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LANGUAGE 
AND THE MIND 


Language is the ultimate human invention. It is the tool that makes 

all other tools possible as well as the most powerful communication 
system in the world. But what gives the human mind the unique ability 
to use language? Where did it come from? How did it get here, both in 
our species as a whole and within each one of us as individuals? These 
are some of the big questions explored in this course, which synthesizes 
research across a wide range of disciplines using a handy framework. 


A common assumption is that human brains must have evolved in 
specialized ways to acquire language. However, current theories suggest 
that this may be exactly backward: Rather than the brain originally 
adapting itself to language, it may be that language adapted itself to fit 
the human brain. 


This course explores that possibility and considers the implications 
for the human mind. It begins by asking what language is and what it is 
not. Language is a complex system of things. Although it shares many 
features with communication systems of other species, language has 
some special properties elevating it into a league of its own. 


This course uses an organizing model known as the 3-D framework. 

For a topic that is so deep and wide, the framework is necessary to 
elucidate how genes, brains, bodies, and environment interact to give 
rise to minds that have language. The framework considers mechanisms 
for language across multiple timeframes and from different angles 

of analysis. 


COURSE SCOPE 


Traditional linguistic approaches have focused mainly on speech as 
the primary medium and monolingualism as the main model of study. 
Although this classic approach has revealed much information, our 
current understanding has benefitted tremendously from taking 

a broader view. This course celebrates language in all its forms: 


¢ The written word. Writing gave birth to modern civilization. 
Because it was invented so recently—slightly more than 5,000 years 
ago—humans are not born with an innate neural module for reading 
and writing. Rather, genes designed for other purposes interact with 
the written word over development, and this constructs a specialized 
reading-writing network in the brain. 


¢ Sign language. Sign language is every bit a language as its spoken 
counterpart. In addition to exploring similarities and differences to 
speech, the course dispels some myths about sign language. 


¢ Embodied language. Many theorize that bodily actions—such 
as hand gestures, eye gaze, and facial expressions—were the 
foundation for language evolution. What aspects of spoken language 
are still grounded in the body, and what parts have transcended 
it? The answer may lead to breakthroughs in robotics, artificial 
intelligence, and neurorehabilitation. 


¢ Multilingualism. Around the world, using more than one language 
is not the exception—it’s the rule. How do multilingual minds 
handle multiple languages, and what are the costs and benefits of 
having them? 


Building on the basic biology that learning language sculpts the brain, 
the course finishes by asking how language shapes other aspects of the 
mind. It considers how having a particular language influences a mind 
specifically as well as how having any language can expand and limit 

a person’s world generally. In the end, the course reveals what makes 
language possible and how it has turned the human mind into the most 
extraordinary creation on the planet. 


COURSE SCOPE 


1. =s 
LANGUAGE IN MIND 


Any product of human civilization was influenced by 
language. It is possible that without language, the human 
mind might never have imagined anything in the first 
place. This lecture first looks at preexisting models of 
the mind. It then presents a model of the mind that will 
be used in the rest of the course, and it closes with a 
look at how language fits within that model. 


Earlier Models of the Mind 


One of the classical models of the mind is dualism, which can be traced 
back to the ancient Greeks. Dualists saw the mind as two distinct entities: 
a physical part, which includes the body and brain, and a nonphysical part. 
For Plato, that was the nonmaterial world of ideas. Later, in traditional 
religious dualism, the nonphysical part was seen as the spiritual soul. 


The most famous dualist of all, René Descartes, took dualism 
out of the religious realm and placed it more in the secular 
world. This type of dualism was called mind-body dualism. 
The body was a machine, and the mind was the home 
of thought, reason, and consciousness. The two entities 
were completely separate, except for where they met 
in the brain through a structure called the pineal gland. 


Around the time Descartes was arguing for his mind- 

body dualism, a rival model of the mind called materialism 

was gaining popularity. Materialism is most associated with Baruch 

Spinoza, Thomas Hobbes, and John Locke, and it blurs the distinction 
between the mind and body. For these philosophers, mind and 

body are one and the same. 


Once the floodgates of materialism were open, it was 

just a matter of time before the mind became an 
object of inquiry for scientists. The first scientist to 
test aspects of the mind in a truly scientific fashion 
was the German physiologist Wilhelm Wundt. 

In 1879, Wundt founded the first official laboratory 

for psychological research at the University of Leipzig, 

and the field of scientific psychology was born. 


Wundt’s approach to studying the mind became known as 
structuralism. Structuralism viewed the conscious mind as composed 

of smaller building blocks of basic sensations and perceptions. Although 
structuralism didn’t last long as a field of scientific psychology, it 
introduced a conceptual approach for understanding complexity at high 
levels—such as conscious thought—by reducing it to much smaller units 
of analysis, including basic sensations and perceptions. 
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The combination of these two things led to a model of the mind called 
biological reductionism. This model views all aspects of the mind 

as grounded in the body, and it dispenses with the idea of the mind 
altogether. There is no mind—only a body. There are only neurons firing 
or genes expressing. This view gained popularity in the 2oth century 
with the incredible advances made in biology, chemistry, and physics. 


In the 21st century, radical reductionism is still embraced by some 
scientists, but it has hit some major roadblocks in its attempt to 
understand the complexities of the human mind. For example, 
reductionists cannot agree on what the lowest level is: Should it be 
neurons, genes, molecules, quarks, or even mathematical equations? 


This Course’s Model of the Mind 


The basic principles behind emergent properties offer a new way 

to think about the mind. An emergent phenomenon might result 

from asynergistic interaction between the brain, the body, and the 
environment. The mind is not reducible to any one of these parts alone; 
it exists only through the combination of them. 


This definition draws from the other models’ strengths while avoiding 
their weaknesses: 


1. Drawing from dualism, we can preserve the mind as a special entity 
without having to commit to a separation from the body. 


2. Drawing from materialism, we can scientifically study the mind 
as we study any other thing without committing ourselves to a 
blank slate. Perhaps aspects of the mind emerge from combining 
innate attributes of the brain and the body with experience from 
the environment. 


3. Drawing from reductionism, we can appreciate the biological 
and physical building blocks of the mind without viewing them as 
the only important items. The other levels higher up may be just 
as significant. 
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Where Language Fits 


Just like the mind, language may be an emergent property that is the 
result of a synergistic interaction between brains, bodies, and the 
environment. One implication of viewing language as something that 
lives partly outside our heads is that we see more clearly how language 
can affect us. If language is similar to other tools we use, it is easier to 
appreciate how it has transformed us and our environment. 


Without the explanatory power of language, it is not possible to keep 
building more and more sophisticated machines. Language is necessary 
to advance from simple tools like levers and hammers to advanced tools 
like computers and satellites. 


In fact, without language, the human mind cannot even conceive of 
creating other tools in the first place. From this perspective, language 
actually transforms the mind to allow it to do things it could not do 
without language. 


The 3-D Framework 


A conceptual model known as the 3-D framework is useful for 
organizing and analyzing the issues discussed in this course. Each of the 
framework’s three dimensions are necessary to understand something 
as complex as language and the mind. The three dimensions are 
causation, levels of analysis, and timeframe. 


Causation concerns two things: What causes a thing, and what does 
the thing cause? This distinction is inspired by Aristotle’s efficient 
cause and final cause. In more modern times, the Nobel Prize-winning 
biologist Nikolaas Tinbergen referred to these two types of causes as 
mechanisms and functions. 


With writing, a mechanism would be the antecedent conditions 
that produce the behavior of writing. An example would be a desire 
to express oneself. In contrast, the function is the consequence 

of writing: making one’s feelings public. For every behavior, there 
are things that produce it (mechanisms) and things it produces 
(functions). 


In the 3-D framework, there are three levels of analysis: biological, 
psychological, and social. The biological level focuses on small things 
like neurons firing and genes expressing as well as larger-scale biological 
activity, such as brain regions communicating and turning thoughts 

into actions. The psychological level concerns subjective experiences, 
like feeling, thinking, and perceiving. The social level concerns how an 
organism interacts with things that are outside the body, which includes 
humans and objects. 


In the example of writing, the biological level concerns brain regions 
that are dedicated to one’s native language. The psychological level 
concerns the desire to express oneself. The social level concerns the 
societal conditions that compel one to write. 


The final dimension of the 3-D framework is the timeframe—that is, 
the scale on which one wants to understand the cause and effect of 
something. It may help to think of this as a timeline from short to long. 
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Here are different increments to keep in mind: 


1. Very brief increments—that is, one millisecond to the next. 
2. Longer increments: one day, week, or year to the next. 
3. Still longer increments: one generation to the next. 


4. Even longer increments: one millennium or evolutionary eon to 
the next. 


This course examines moment to moment, developmental, historical, 
and evolutionary timeframes. Approaching a subject from these 
different timeframes fundamentally changes how we think about them. 


Returning to the example of writing, the smallest timeframe—the 
moment to moment—involves the immediate antecedents to putting pen 
to paper: neurons firing to link ideas into words or a flash of inspiration. 
The developmental timeframe concerns slower mechanisms, such as how 
a lifetime of using language has wired the brain to think in words. 


The historic timeframe considers the larger context in which a behavior 
occurs. This context involves large concepts like cultural institutions, 
national identity, and technological innovations. 


Finally, the evolutionary timeframe involves chunks of time millions of 
years long, a scale that differentiates one species from the next. This 
timescale makes it possible to discuss the unique genetic baggage that 
different species carry around with them. Particularly relevant is the 
fact that humans have a set of genes and some basic learning abilities 
that endow us with the capacity to write. 


Bringing It Together 


The three dimensions of the framework interact with one another. For 
example, a mechanism can have three levels, and each level can span 
four timeframes. As an example of this in action, consider this apparent 
contradiction: If reading and writing are innate, how can we not be born 
with a specialized brain mechanism dedicated to them? 
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Levels of Analysis 


Mechanism Function 


The 3-D framework offers a way out. The only way to solve this 

puzzle is to consider mechanisms for reading and writing on different 
timeframes and multiple levels of analysis. To begin, on an evolutionary 
timescale and a biological level, humans are genetically equipped with 
brain regions that are designed to do very general things (like seeing 
objects, hearing sounds, controlling motor movements, and making 
associations). Humans are also evolutionarily endowed with a brain that 
is highly plastic. Neuroplasticity means that the brain can change itself 
in response to the environment. 


On a developmental timescale and a social level, human children are 
massively exposed to spoken and written language throughout their 
formative years. Before children begin to read and write, most have 
heard, used, or seen many millions of words. 


Here is where the two timeframes and two levels meet: The massive 
social exposure of words over development combines with the brain’s 
innate potential to change itself. In turn, this transforms brain regions 
that were evolutionarily designed for more general things like vision, 
hearing, motor movements, and making associations. It transforms 
these regions into a specialized reading and writing device. 
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In this way, reading and writing are emergent properties of the mind. 
Part of this is innate and controlled by genes, but another part is the 
result of prolonged experience with the environment. It’s not possible 
to explain reading and writing by focusing only on one timeframe 

or only a single level of analysis. The key is to understand how these 
dimensions interact. 


Suggested Reading 
Sapolsky, Behave. 


Question to Consider 


Using the 3-D framework, can you dissect and analyze an aspect 
of the mind that is particularly interesting to you? Here are a few 
sample topics: perceiving human faces, mathematical thinking, 
musical ability, psychological disorders, and bilingualism. 
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2. 
LANGUAGE AS A SYSTEM 


A very basic but important question is this: What is 
language? To begin tackling that question, this lecture 
defines five. components of language. Those components 
are syntax, semantics, morphology, phonetics, and 


OO0OU OF 
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Syntax 


Syntax primarily concerns the rules for 
how a language orders words in a sentence. 
An example of the default structure in 
English is this sentence: “The child opened 
the door.” This follows a form called SVO 
syntax, with SVO standing for subject, verb, 
and object. The child is the subject, opened 
is the verb, and the door is the object. 


Other languages order sentences 
differently. Forty-four percent use a 
subject-object-verb pattern, including 
Japanese, Turkish, and Hindi. Additional 
setups exist as well. The takeaway point is 
that there are multiple ways for syntax to 
function across languages. 


Semantics 


SVO Languages 


More than 
40 percent of 
all languages 

conform to the 


SVO pattern. 
English, Spanish, 
German, and 
Mandarin are 
well-known 
examples. 


In linguistics, semantics is the study of word meanings. More technically, 
it is the study of symbols and their referents. One of the most striking 
facets of semantics is that the meaning of each word is traditionally 
defined by other words. This makes semantics an endless cycle of 


symbols referring back to themselves. 


Additionally, linguistic symbols differ across all languages. Some of 
this variation is in sheer number: One Korean dictionary lists more 
than 1 million words, whereas Oxford’s classical Latin dictionary lists 


roughly 40,000. 


The semantic component of language is notable for being asystem 
within a system. There are different word types that have quite different 
properties. For example, consider open- and closed-class words. 
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Open-class words form a category of words that can be changed, 
added, or removed from a language. In English, these are nouns, verbs, 
adverbs, and adjectives. For instance, the word trousers has largely 
faded into obscurity, but words like email have come into existence. In 
contrast, closed-class words are words that form a more fixed category 
that is not easily altered. In almost all languages, these are prepositions, 
articles, and pronouns. 


Morphology 


The next component is morphology. This linguistic component is made 
up of morphemes, which are the smallest meaningful grammatical units 
of language. Take this sentence as an example: “I’m getting really hot.” 
The word hot is a single open-classed morpheme, but every other word 
in the sentence has two morphemes. /’m is a contraction of / and am. 
The word getting uses the verb stem get plus -ing, which is how English 
indicates an activity happening in the present. And the term really is 
formed with real plus -ly, which is how English turns an adjective into an 
adverb. Note that -ing and -ly are closed-class morphemes. 


Languages vary in how many morphemes they pack into a word. 
English is relatively light on morphemes. Most words are one or 

two, but certain words have many more. For example, the word 
antidisestablishmentarianism was created in 19th-century Britain to 
protest attempts to remove the Church of England as the official state 
religion. It has six morphemes. 


Phonetics 


The smallest part of language is the phonological or phonetic 
component. Spoken words and morphemes are all made up of 
phonemes, which are the most basic meaningful sound units of a 
language. In a sign language, they would be the smallest manual units. 
English has a total of 44 phonemes. 
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Within a language, there are rules about which phonemes can and 
cannot follow one another. For instance, while Polish allows many 
consonants to cluster without a vowel, English permits far fewer. 
English allows three at most at the front of a word, as in splash, and 
four at most at the end of a word, as in twelfths. 


As with morphemes, there is a large range of phonemes used in 
different languages. On the low end, the Amazonian language of Piraha 
is thought to have only 11 phonemes, and on the other end, there are 
African click languages that have up to 141 phonemes. 


Pragmatics 


The final component of language, pragmatics, refers to how people use 
language, and it reveals language as a powerful tool for accomplishing 
social goals. In the study of pragmatics, these goals are often referred to 
as intentions, and understanding intentions can be a tricky business. 


For example, take this sentence again: “I’m getting really hot.” There 

are many potential meanings of this utterance. One meaning is a literal 
statement about a person’s rising temperature. Another meaning could 
be that the person is becoming angry. Yet another option is that it could 
be an indirect request to turn on the air conditioning. 


There is often nothing inherent in the literal meaning of the words that 
reveals a speaker’s intentions. To solve this problem, a listener needs to 
make a pragmatic inference about the intentions of the speaker. 


Because pragmatics involves social goals, it interfaces with culture in 
ways that other parts of language do not. In fact, pragmatics is the one 
component of language that can get people in the most trouble when 
communicating with people from other cultures. For instance, many 
Americans love sarcasm, but it can come across as confusing or rude in 
some other cultures. 
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Conclusion 


Language is nonlinear and multidirectional. In addition to piecing the 
smallest units into larger ones, language works in the other direction 
too: The largest units work backward to simultaneously guide the 
processing of the smaller ones. 


For example, pragmatic goals and intentions are often the starting point, 


not just the ending point, for all other parts of language. Recall that in 
the example, “I’m getting really hot,” the meaning of the utterance is 
largely dependent on the context. 


Another example comes from when the syntax of a sentence sets 

the listener or reader up to expect a certain meaning. Consider this 
utterance: “Time flies like an arrow, but fruit flies like a banana.” The 
syntactic trick here is that flies in the first part of the sentence is a 
verb, but in the second part, it is a noun. These are called garden path 
sentences because they lead the listener or reader down one syntactic 
path, only to play a trick with a different path later. 


Suggested Reading 
Sedivy, Language in Mind. 


Warren, “Perceptual Restoration of Missing Speech Sounds.” 


Questions to Consider 


Can you think of other ways that different components (levels) 

of language interact to allow language to function properly? Over 
development or when learning a second language as an adult, how 
might some components of language facilitate other components? 
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3- 
ELEVEN LINGUISTIC 
UNIVERSALS 


This lecture presents a list of 11 linguistic universals 
that are true of all human languages. Nine of these 
universals are drawn from what the American 

linguist Charles Hockett called basic design features 
of language. The lecture also adds two additional 
features that reflect recent advances in developmental 
psychology and cognitive neuroscience. 
Together, these 11 characteristics form 
a good starting point for understanding 
what language is and what it is not. 
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Feature 1: Duality of Patterning 


The first feature, duality of patterning, refers to language having 

a double structure. Smaller units like phonemes possess no inherent 
meaning, but these combine to create meaning at higher levels, like 
morphemes, words, and sentences. 


Feature 2: Arbitrary Symbols 


Arbitrary symbols are the feature that has received the most 
attention from linguists, philosophers, and scientists. In spoken 
language, the relationship between a word (or a symbol) and its 
meaning (or its referent) is largely arbitrary and based on the quirky 
conventions of particular languages. 


However, many words possess a feature that cognitive linguists 
call iconicity. For example, onomatopoeias are classic examples of 
iconicity. The words buzz and bang sound like what they mean, to 
some extent. 


Feature 3: Language Is Rule Governed 


All languages are rule governed. The rules are hierarchical in nature, 
which means that rules of larger units of language are built up from 
smaller and smaller units. This refers back to the idea of duality of 
patterning. For example, the syntax of a language works only if the 
rules of semantics are in place. Semantics rely on rules of morphology, 
and these morphemes require that the rules governing phonemes 

be followed. 


Feature 4: Generativity 


In the late 1950s, the linguist Noam Chomsky developed a theory that 
children learn language because they are born with an innate set of rules 
that allows them to create novel linguistic utterances. Chomsky called 
this the generativity of language, and Hockett later called it productivity. 
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The idea is that people can generate or produce 

an infinite number of new things that they have 

never heard simply by using words in a rule- 
governed way. 


For instance, at the morphological level, 

children are so creative that they often 

make up words that don’t exist in their 

native language, like pluralizing the word 
fish as fishes. This ability seems to be 
uniquely human. 


Feature 5: Displacement 


Displacement refers to the fact that language can be used to 
communicate about things not in the here and now. For instance, 
children talk about topics that are displaced when they ask questions 
such as “How many days until my birthday?” 


This sort of communication is quite rare in the animal kingdom. 
However, there are some well-known exceptions of animals 
communicating outside of the here and now. For example, bees 
communicate about the precise location of far-off food sources. 


Feature 6: Cultural Transmission 


Cultural transmission means that human language is not directly and 
completely passed down through our genes. Nobody is born with a 
brain designed to speak Portuguese, Bantu, or Hawaiian. To speak these 
languages, we need to learn them first. 


This should not be taken for granted, because most other species are 
different: They are born with an innate and fixed set of communication 
signals, so there is no learning required. This is not true of all animals— 
for example, songbirds are a notable exception—but it is true for 

most animals. 
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Feature 7: Reflexiveness 


Perhaps the most interesting and unusual topic that humans talk about 
is talk itself. This feature is called reflexiveness: Human language can be 
used to describe or refer to itself. This occurs in many ways. A simple 
example is “I didn’t hear you.” 
There are also more complicated 
examples. For instance, the 

legal profession is built on the Versatility of Human 
interpretation of words. Communication 


Communicating about 
communication is what cognitive 


The features of 


psychologists call metacognition, displacement, cultural 
and it goes hand in hand with transmission, and 
another unique aspect of the reflexiveness illustrate 


human mind: Humans can think the remarkable range of 
about thinking. This is one of the mencana 
crown jewels of human cognition. Absolutely nothing is 


off-limits for humans. 
Feature 8: Specialization 


Language is specialized for 

one purpose: to deliberately 
communicate. This may seem self-evident, 
but consider that other behaviors can 
communicate without being specifically 
designed for it. For instance, imagine 

a person cursing after stubbing his 

toe. Was he trying to communicate 
something, or was that just a reflex? 
Flirting, too, can be unintentional. 


Those examples are murky, but language 
is crystal clear. When a person says a word 
or produces a sign, he or she has one purpose in 
mind: to communicate. 
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Feature 9: Prevarication 


The ninth design feature is prevarication, which means that human 
language allows us to lie. The fact that humans lie seems bizarre given 
the semantic property of language. Humans put great effort into 
creating precise words for things, so why would we not accurately 
describe those things with the right words? The answer comes from the 
pragmatic component of language: Pragmatics tells us that words are 
driven by intentions, and lying with language can help actualize goals. 


Feature 10: The Standard Brain Network 


The final two items are add-ons to Hockett’s original list of design 
features. Rather than being features of language per se, these are 
universal patterns of how language is organized in the brain and how it 
unfolds during development. There is remarkable similarity in the neural 
machinery for all language users on the planet. The brain is divided into 
two hemispheres, the left and the right, and four lobes: the frontal, 
temporal, occipital, and parietal lobes. 


For most people, the left hemisphere A ~~ 
contains brain regions that are lin a 
specialized for language. For example, E 
there is a region in the posterior part A 
of the left hemisphere frontal lobe ) 
called Broca’s area that is traditionally | 
associated with language production,  Boesaea © E = 
Farther back in the superior 
temporal lobe is Wernicke’s area, 
which is traditionally associated with 
language comprehension. 


>», Wemicke’s area 


This neural organization is uniform across brains. Although a small 
percentage of people have their Broca’s and Wernicke’s areas in the 
right hemisphere—and some even have it distributed across the left 
and right hemispheres—there has never been a case in which the two 
areas are entirely absent in someone who uses language. 
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Feature 11: The Shared Developmental Trajectory 


The final universal has to do with language acquisition. Across all 
languages, there is a shared developmental trajectory, especially within 
the first year of life. 


The first shared aspect of language development is that all newborns 
possess an innate ability to perceive phoneme contrasts in all languages. 
We know this based on classic work done by Janet Werker from the 
University of British Columbia in the 1970s. 


Over the first six months of life, babies focus their attention on the 
phonemes that are the most relevant to their native language. Following 
this perceptual focusing, babies begin to practice producing these 
phonemes in the form of babbling. These babbles are not restricted to 
speech sounds. Deaf babies who are exposed to sign language as their 
primary language babble with their hands. 


The next stage is that children start to understand their first words at 
six to seven months of age. Shortly after this, babies start to point at 
things in their environment. Researchers believe that this is an attempt 
to communicate one’s attention toward and intentions about objects in 
the world. Finally, by roughly their first birthdays, most babies start to 
say their first recognizable words. 


Suggested Reading 
Hockett, “The Origin of Speech.” 


Questions to Consider 


To what extent are the design features of language unique to 
language? How might they actually reflect more general aspects of 
the human brain? 
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4. 
COMMUNICATION IN 
THE ANIMAL KINGDOM 


To understand how language is uniquely sculpted to 

the human mind, it is necessary to use the 3-D framework to 
explore the biological, psychological, and social prerequisites 
that make humans the perfect fit for language. This lecture 
explores the psychological and social commonalities and 
differences in communication and cognition between 
humans and nonhumans. 


Honeybees 


The first place to look for commonalities and differences is in the 
natural communication systems of nonhuman species in the wild. 
Modern research in the areas of ethology (the study of animal behavior) 
and comparative psychology has greatly benefited from methodological 
advances in audio, video, and computer technologies, and we now know 
more about nonhuman communication than ever before. 


One notable species is the honeybee. These bees have a very impressive 
communication system for letting other bees know about the location 
of food sources, such as flower nectar. When a worker bee finds a food 
source, it rushes back to the hive and performs an elaborate dance on 
the surface of the honeycomb. This dance communicates information 
about the direction, distance, and quality of a food source. 


The dance of bees and the language of humans differ in some significant 
ways. To start, unlike human language, a bee dance is an innately 
specified system that is not learned by observing others. Additionally, 
the elements of dance may not be truly symbolic in the same way as 
human language. Finally, bees dance only about food and water. This 
suggests that the function of their dance is fixed and narrow. 


Vervet Monkeys 


One of the classic studies on primate vocal 
communication was done with vervet 
monkeys by three researchers in 1980. 
Vervet monkeys are small, gray-haired 
monkeys that live in groups in woodland 
and mountain areas of East Africa. They 
have three primary predators: leopards, 
eagles, and pythons. These predators are 
so dangerous that vervet monkeys have 
developed vocal alarms that let the rest of 
the group know of their presence. 
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The calls are specific to the three types of predator. The researchers 
found that leopard alarms cause monkeys to run for cover in trees, 
eagle calls caused them to run for cover in rocks or low bushes, and 
snake alarms caused them to search the ground around them. 


Follow-up studies showed that this skill was not entirely innate. Baby 
vervets learned how to make the appropriate calls by observing adult 
monkeys produce the calls. 


Chimpanzees 


Chimpanzees in the wild communicate with vocalizations and hand 
gestures. The pioneering primatologist Jane Goodall was one of the 
first scientists to document that chimps produce various types of bodily 
gestures when interacting with other chimps. 


In one of the largest-scale studies done in the wild, Catherine Hobaiter 
and Richard Byrne recorded more than 4,500 gestures produced by the 
Sonso community of chimpanzees from the Budongo Forest in Uganda. 
These gestures were produced by dozens of chimpanzees, both adults 
and children, and there were more than 60 distinct meanings recorded. 


The takeaway point is that it appears that other animals can use 
arbitrary symbols to deliberately communicate meanings with other 
members of their species. However, it is worth pointing out an aspect of 
this sort of nonhuman communication that is different from language. 
There is very little evidence that these animals naturally string together 
more than one or two calls or gestures in rule-governed ways to 
produce novel sentences. 


Teaching Language to Dogs 


This lecture now turns to the question of whether it is possible to 
teach language to nonhumans. There have been countless natural 
experiments tried with humans’ best friend: dogs. 
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It is clear that dogs can learn basic words, including sit and stay. 
Additionally, there are many cases of dogs that have vocabulary of 
dozens and even hundreds of words. The most famous and impressive 
example is Chaser, a border collie trained by the psychology professor 
John Pilley. Chaser was said to know more than 1,000 words for 
commands, toys, and objects, which is comparable to a typical human 
three-year-old. 


One key difference between a word-learning dog and a typical child is 
the inferences they make when learning new words. When a child learns 
a word, they implicitly categorize those words based on certain features, 
particularly the shape of an object. 


Matters are less clear with dogs. Emile van der Zee performed 
controlled experiments on a different border collie named Gable and 
found a different type of categorization: the size of an object. Even that 
was not stable from one learning trial to the next. Compared to dogs, 
humans seem to have very specific and consistent cognitive heuristics 
that guide their early word learning, suggesting that they start with 
some quite distinct cognitive machinery. 


Dolphins and Language 


Dolphins naturally communicate by whistling, 
which some believe has symbolic properties. 
For example, dolphins seem to have distinct 
whistles for particular referents. Building 

on this, there have been attempts to teach 
dolphins an even more elaborate language-like 
communication system. 


The most extensive investigation was done by Louis Herman, who 
conducted systematic tests of bottlenose dolphins at the University 
of Hawaii for more than three decades. Herman’s two most famous 
dolphins were Phoenix and Ake. Phoenix was exposed to an artificial 
tonal language that was transmitted by computer through speakers 
broadcasting underwater. 
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Phoenix was able to learn an impressive number of tonal words, and 
she could also understand basic syntactic distinctions. For example, she 
could differentiate between “take the ball to the hoop” and “take the 
hoop to the ball.” 


Phoenix’s comrade, Ake, was just as impressive. Unlike Phoenix, Ake 
was trained to respond to videos of hand gesture commands with 
underwater button presses. (Dolphins press buttons with their noses.) 
Perhaps Ake’s most striking linguistic feat was to correctly respond to 
questions about remembering objects that were not physically present. 


In one experiment, when asked if a hoop was or was not located ina 
different tank that Ake could not see, she was able to answer correctly 
more than 90 percent of the time. This is one of the few examples of 
displacement in nonhuman communication. 


Additional Work on Primates 


Dolphins are impressive, but the most promising bet for teaching 
language to animals comes from work on primates. Much of the modern 
work on teaching language to primates can be traced 
back to a paper written in 1909 by the American 
psychologist Lightner Witmer. He inspired a 
century of research on primate language 
with his paper titled “A Monkey with a 
Mind.” 


The paper focused on Peter, who was 

a three-year-old captive chimpanzee. 
(Despite the paper’s title, he was an ape, 
not a monkey.) Peter had been impressing 
audiences in the Vaudeville circuit for his 
remarkable humanlike behaviors. 


Witmer was so intrigued by Peter that he asked his trainer if he could do 
scientific tests on Peter’s cognitive abilities. Peter passed many of the 
complex tests Witmer threw at him, but Peter’s language skills were not 
quite as impressive. 
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Peter could understand basic commands, but Witmer wondered 
whether Peter could have learned more had he been exposed to 
language at a younger age. This simple question set the stage for 
language-training studies on primates for the next 100 years. 


Many of the early attempts to test this hypothesis took an extreme 
approach: Researchers adopted baby chimps and tried to raise them 
as human children. However, in the late 1960s, researchers took a new 
approach to immersing primates in language. Influenced by research 
showing that sign language shared all the basic universals of spoken 
language, researchers moved away from vocal training to training of 
hand gestures, or signs. 


One of the most famous cases is Washoe. Washoe was studied by a 
husband-and-wife psychologist pair, Allen and Beatrix Gardner. Adopted 
as a two-year-old, Washoe was taught American Sign Language (ASL) 
as her exclusive form of communication. Washoe eventually learned 
more than 350 novel ASL signs. She could understand and produce sign 
strings, and in some cases, she could communicate about things not 
present in the here and now. 
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Challenges to the Work 


There are other well-known examples of language learning in primates, 
such as Kanzi the bonobo and Koko the gorilla. However, there are also 
major challenges to this line of work. 


First, many scientists have argued that there is a high degree of so- 
called rich interpretation of the data. Language-training studies require 
an immense amount of time, energy, and resources, and this makes it 
hard for the researchers to see things objectively. 


A related critique concerns the nature of what is actually learned by the 
animals in these studies. Essentially, the question is this: Are primates, 
dogs, and dolphins really learning to communicate like humans, or 

are they just learning to produce complex behaviors in response to 
various cues? 


Suggested Reading 


» 


Gardner and Gardner, “Teaching Sign Language to a Chimpanzee. 


Seyfarth, Cheney, and Marler, “Monkey Responses to Three 
Different Alarm Calls.” 


Questions to Consider 


Do you think that scientists have not been creative enough in 
testing whether nonhuman species are capable of language? Have 
they been too anthrocentric in imposing a human communication 
system onto a nonhuman mind? Can you think of a better 
approach? 
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5e 

GENES, 
BRAINS, AND 
EVOLUTION 


Being exposed to 
language is not 
enough to ensure its 
development. This 
lecture explores the 
genetic and biological 
landscape that allows 
language to thrive 

in humans but not 
other species. 


Basic Background Information 


The latest estimates show that our bodies contain approximately 
35 trillion human cells. (Aside from human cells, there are other 
cells present, which are largely bacteria.) Almost all human cells 


contain along, coiled molecule of 

DNA that is organized according to 23 
pairs of chromosomes. Each of these 
chromosomes contains hundreds or 
thousands of individual protein-making 
molecules called genes. In total, we have 
roughly 20,000 genes. These genes 
provide the basic instructions of what each 
cell is supposed to do. 


The concept evolutionary conservation 
means that basic genetic mechanisms are 
reused by a large number of species. Often, 
these are biological processes that serve 
basic survival functions, ranging from the 
small—like ion channels that modulate cell 
activity—to the large, such as important 
organs. This shared equipment is worth 
highlighting because it suggests that the 
features unique to particular species need 
not require completely new genes. 


Overlap 


Within the 
human 
species, there 
is remarkable 


overlap of 
genetic material: 
We share about 
99.5 percent of 
DNA with one 
another. 


This logic runs counter to commonly held intuitions about language. 
Because language is so complex and so novel, it seems logical that 


there must be a special set of genes dedicated to it. 


However, recall that language is not a single thing: It is a system of 
things. Evolution may have selected for the system, but parts of the 


system were borrowed from genetic material shared with other species. 


This is the basic idea of evolutionary conservation: Evolution often 


tinkers with old things to make new ones. 
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As it pertains to language, this view was best summed up by the 
cognitive scientist Elizabeth Bates. Bates did not see language as a new 
machine built out of new parts. Based on the principle of evolutionary 
conservation, Bates viewed language as a new machine built out of 

old parts. 


There are two main genetic arguments to support Bates’s view of 
language. The first is that there are no single genes for any complex 
behavior. Just like there will never be a gene for empathy, schizophrenia, 
or belief in God, there will never be a gene for language. 


Second, there is growing evidence from molecular biology that genes 
involved in language are not unique to humans. As with other cases 
of evolutionary conservation, the genes used for language are shared 
among many species. 


FOXP2 


One particular gene has received the most attention in the molecular 
study of language: FOXP2. The story of FOXP2 starts with a British 
family in the 1980s, medically dubbed the KE family. Members of this 
family were observed to have an unusually high preponderance of a 
particular language disorder called developmental verbal dyspraxia. 


One of the defining symptoms of this disorder is a difficulty combining 
phonemes while saying words. The disorder in the general population 

is quite rare, but across three generations of the KE family, about 50 
percent of members had some form of it. This led to the hypothesis that 
there may be a genetic mechanism underlying the problem. 


In 1998, neuroscientist Faraneh Vargha-Khadem, geneticist Simon Fisher, 
and their colleagues published a paper locating the problem ona stretch 
of genes on chromosome 7. Later, in 2001, using more powerful gene 
imaging techniques, the exact location of the mutation was identified 

on the FOXP2 gene. Later still, it was discovered that the problem 

was caused by a modification to just a single letter of the FOXP2 DNA 
sequence, which was traced back to a mutation passed down by the 
grandmother of the KE family. 
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The FOXP2 gene produces a protein called the FOXP2 protein. The 
human version of this protein differs by only two (out of 175) amino 
acids from the chimp version and by only three from the mouse version. 
This protein acts on various other cells during prenatal development 

to affect not just the brain but also other organs, such as the heart 

and lungs. 


Although the FOXP2 gene is implicated in many aspects of an organism’s 
development, its effects on the brain are what directly connect it to 
language. In humans, the brain regions that are affected downstream by 
the FOXP2 mutation are areas involved in motor control of the mouth 
and tongue, which are clearly necessary for speaking. 


Using state-of-the-art gene-editing techniques, scientists now have 
the capability to extract segments of human genes and insert them 
into the genome of other animals. For example, Wolfgang Enard has 
done research that snips the human mutation of FOXP2 and places it 
into the appropriate segment of the genome of living mice. The result 
is significant disruption to motor brain regions involved in controlling 
high-pitched vocalizations, which is how baby mice communicate with 
their mothers. 
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Enard also describes the opposite technique: inserting a fully 
functioning human FOXP2 gene into a mouse genome and testing 
what happens. Remarkably, this resulted in a proliferation of neuronal 
connectivity in motor regions of the mouse brain. Research is ongoing. 
If these changes correlate with enhanced vocal motor performance 

in the genetically altered mice, it would suggest that the human 
variant of the FOXP2 gene may have indeed been an evolutionary 
springboard from more simple communication to the complexities 

of language. 


Beyond FOXP2 


Beyond FOXP2, there is hope that we will discover new gene variants 
that connect to different aspects of language. Language is a complex 
system. The FOXP2 gene links mostly to phonological and some 
syntactic aspects of language, but other aspects of language are 
unrelated. Perhaps we will find genes that map onto these other parts 
of the system. 


There are cases of other genetic disorders where some aspects of 
language are disrupted, but others are spared. For example, Williams 
syndrome is a genetic disorder caused by the deletion of 26 to 28 genes 
on the seventh chromosome. This causes a unique psychological and 
social profile. 


Carolyn Mervis of the University of Louisville did some of the 
pioneering work on Williams syndrome, and she describes a disorder 
where there are severe cognitive deficits in math and spatial abilities but 
a relative sparing of some language abilities. In particular, phonological 
and semantic abilities are close to average in these individuals. However, 
their syntactic abilities are lower than their phonological and semantic 
skills, and their pragmatic abilities are extremely compromised. 


These pragmatic deficits are reminiscent of problems also seen in 
individuals with autism spectrum disorder (or ASD). The National 
Institutes of Health describes children with ASD as “often socially 
withdrawn” and as having trouble with verbal communication. 


5. GENES, BRAINS, AND EVOLUTION 


33 


However, some people with ASD have incredibly rich vocabularies 
and very sophisticated syntactic skills. This suggests an interesting 
dissociation between semantic and syntactic abilities from 
pragmatic skills. 


To sum up, here are a pair of takeaways from this section of the lecture: 


+ Because language is a complex system, we must recognize that 
different aspects of language are affected by different genes and 
gene networks. 


¢ Itis likely that much of the machinery of language is built on genes 
that are shared with other species. 


Homologies in Brain Structure 


A homology is a structure or a process that is shared among species due 
to a common evolutionary ancestor. They can be at the gene level, as 
with FOXP2, and also at a higher level of organization, such as cell types 
or organs. 


Humans and many other animals share homologies related to language. 
For example, the chimp brain and human brain are anatomically very 
similar. We both share the basic hemisphere and lobe structures. 

Even Broca’s and Wernicke’s areas have homologous structures in 

the left hemisphere of chimps. Just as with humans, these regions 

are involved in the production and comprehension of vocalizations. 


Even though there are many homologies between humans and 

our closest primate cousins, there are some major differences. 

For example, a human brain is much bigger than a chimpanzee brain, 
even when adjusted for relative body weight. Recent research by a 
leader of the Human Genome Project, David Haussler, suggests that 
these differences may be driven by a unique family of genes called 
NOTCHA2NL. These genes emerged after humans split from the ape 
lineage several million years ago. 


This genetic difference may also be responsible for the number and 
complexity of neural connections in the human brain. One recent 
estimate is that there are 100 trillion connections. 
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Neural Connections 


More than sheer number, the way neurons are connected is crucial. One 
of the novel aspects of the human brain is how many connections it has 
between new and old brain structures. 


In all mammals, the newest part of the brain, called the neocortex, lies 
on the surface of the brain. Older parts, like the subcortex, lie deeper 
beneath the surface. Both structures are very old: It is estimated 

that the neocortex was added on top of the subcortex when our 
distant mammal ancestors diverged from reptiles during the time of 
the dinosaurs. 


The neocortex is where most high-level cognition occurs. That is where 
Broca’s and Wernicke’s areas are located in humans. It is also where 
the executive functions are housed, mostly in the frontal lobe. These 
executive functions in the frontal lobe are involved in monitoring and 
regulating thoughts and actions. 


In contrast, the much older subcortex is where many basic survival and 
emotional centers are located. One of the functions of having more 
neural connections between the frontal lobe and the subcortex is to 
have an increased ability to modulate and regulate basic impulses and 
drives. In general, this executive processing function of the frontal 
cortex is one of the most important attributes of all mammal brains and 
human brains in particular. 


This is relevant to language when it comes to the differences in what 
species communicate about. Most animal communication is about 
basic topics such as food, territory, protection, and sex. These are all 
primal instincts that have direct links to survival, either immediately 

or in the long run. The primary neural mechanisms underlying these 
drives are located in a subcortical brain network called the limbic 
system, which contains structures such as the hypothalamus, amygdala, 
and cingulate. Having weak connections from the frontal lobe to the 
limbic system allows these lower and older brain regions to drive and 
dominate communication. 
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Cingulate gyrus 


Hypothalamus 


Amygdala 


Matters are different in humans. Because we develop much stronger 
connections between the frontal cortex and the limbic system, humans 
have much more control over what they communicate about. Humans 
still have plenty of the basic limbic system urges, but our frontal lobes 
have liberated us from constantly focusing on them. In addition to 
freeing us up to talk about a wider range of things, the frontal lobe has 
helped us learn how to be more reflective about our behavior and the 
behavior of others. 


Suggested Reading 


Fisher and Vernes, “Genetics and the Language Sciences.” 


Questions to Consider 


Elizabeth Bates makes this claim: “Language is a new machine built 
out of old parts.” What other aspects of the human mind might 
also be constructed this way? In contrast, what aspects of the mind 
might be new machines built out of new parts? 
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6. 
HOW THE BRAIN 
CREATED LANGUAGE 


There are many theories of language 
evolution. This lecture focuses on 

a broad conceptual divide between 
two competing ways of thinking: 

the nativist camp and anti-nativist 
camp. The nativist camp believes that 
language is the product of biological 
evolution and that human brains at 
birth are genetically endowed with 
specific abilities to learn language. 
The anti-nativist camp argues that 
language is the product of cultural 
evolution and that human brains at 
birth are genetically endowed with 
general abilities to learn language. 


Everyone in this debate agrees that 
there is a unique and special fit 
between language and the human 
brain. The main disagreement is in 
what initially drove this relationship: 
Did the brain adapt to language, or did 
language adapt to the brain? 


The Nativist Proponents 


Proponents of the nativist camp include linguists like Noam Chomsky 
and psychologists like Steven Pinker. They believe that the human brain 
gradually evolved something called a mental organ that was specifically 
designed for language. 


The idea is that at some point in our evolutionary past, there was 

a genetic mutation that changed our ancestor’s brains, transforming 
their regular communication into something more like language. 
Because this type of communication was so adaptive and so key to the 
survival of the individuals who used it, it rapidly spread language genes 
throughout the population from one generation to the next. 


The Anti-Nativist Proponents 


In contrast, the anti-nativist view approaches the relationship of 
language and brain in a totally different way. Morten Christiansen and 
Nick Chater’s theory of language evolution turns the language/brain 
relationship on its head. Instead of viewing language as shaping the 
brain, they see it the other way around: The brain has shaped language. 


Christiansen and Chater point out that a fundamental tenet of natural 
selection by evolution is that organisms need stable environments 

to adapt to. Because it can take millions of years for an organism to 
radically change itself, it requires that the new environment does not 
also change. The theory suggests that language is not part of our 
genetic endowment at birth but rather a cultural invention of our 
unique brains. 


Reconciling the Views 


Nativists and anti-nativists agree that the human brain is genetically 
endowed with some impressive cognitive and social skills. These skills 
apply to more than just language; for example, they might have helped 
our ancestors in domains such as hunting and building. In psychology, 
skills that serve multiple functions are called domain-general. 
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One likely scenario is that humans possessed some powerful domain- 
general skills in our evolutionary past that they co-opted to create 
language. Perhaps some individuals in a community started to apply 
their general skills of problem solving, pattern recognition, and 
perspective taking to how they communicated with others in their 
community. Over time, from one generation to the next, it is not 

hard to imagine how these communication skills could have gradually 
transformed to become more and more like what we now call language. 


This is the genius of Christiansen and Chater’s theory: There is no need 
to posit any special genes for language. Language could have arisen out 
of old genes that were simply repurposed for language. 


However, one large question is this: If language didn’t arise out of 

a specific genetic change to the brain, what general brain changes 
gave birth to language? Scientists are a long way from answering this 
question. Scientists do not even agree when language first evolved: 
Estimates are anywhere from 100,000 years ago to 4 million years ago. 


Without knowing the timeline, it is very hard to know what changed 
in the brain to enable language. There are many theories, but most 
of them share one common denominator: the prefrontal cortex. 
Archaeological records suggest that over the past 2 million years, 
the human brain has tripled in size, and much of that growth was 
concentrated on the frontal lobes. Connectivity plays a role as well. 


Putting this all together, one reasonable hypothesis is that the frontal 
lobes of our prelanguage ancestors evolved to become better problem 
solvers, pattern recognizers, and perspective takers. These general skills 
supercharged our communication system, giving it the spark that would 
eventually turn it into language. 


Yet another question emerges, though: How did domain-general skills of 
the brain help our ancestors turn general communication into what we 
now call language? To put it in Christiansen and Chater’s terms, how did 
the brain shape language? Four lines of evidence provide clues into how 
this may have happened. 
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Evidence Line 1: Written Records 


First, we can use written records to understand how languages have 
changed over time. One way languages change over time is that they 
become more learnable. When a language propagates over several 
generations, many aspects of the language that are hard to learn 
become less prominent, and many aspects that are simple to learn 
become more prominent. Along these lines, languages change to 
exploit the strengths of human cognition. For instance, the fact that 
humans are good at taking different perspectives allows our pragmatics 
to become less blunt and more refined. The takeaway point is that 
languages don’t change randomly. They are constrained by the people 
using and learning them. 


Evidence Line 2: Modern-Day Language Creation 


A second set of clues comes from modern-day language creation. 

Take, for example, the Al-Sayyid Bedouin community, which comprises 
about 3,500 people living in an isolated village in the Negev desert. 
Largely because of their isolation, they have had an unusually high rate 
of congenital deafness since the mid-1930s. This makes for a fascinating 
question: How do these deaf individuals communicate? 
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Al-Sayyid Bedouin community . 
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That question was answered by a team of international researchers: Mark 
Aronoff and Carol Padden in the US and Irit Meir and Wendy Sandler in 
Israel. The answer is that they have invented their own language called Al- 
Sayyid Bedouin Sign Language. This is a fully functioning language that has 
evolved over time. This language’s establishment and evolution suggest 
that some general skills may be innate, and learners may use those to 
mold the language as they acquire it. In this way, the brain is shaping 

the language. 


Evidence Line 3: Computational Linguistics 


A third line of work comes from an interdisciplinary field called 
computational linguistics. This field brings together computer scientists, 
philosophers, psycholinguistics, anthropologists, and neuroscientists 
who use computers to simulate language development, evolution, 

and change. 


One of the leaders of this field, Simon Kirby of the University of 
Edinburgh, did a series of studies in which he created a computer 
program that taught virtual learners novel symbols and meanings over 
many trials through a process of iterated learning. 


Eventually, there emerged a systematic organization that looked a lot like 
language. From these results, Kirby concluded that the virtual learners 
imposed organization on what they were learning at every iterative step 
to make it increasingly easier to learn for the next generation. 


Although a far cry from real language evolution, this is a useful existence 
proof. It shows that linguistic complexity can arise from very humble 
beginnings. It also suggests that languages are dynamic and change even 
if learners do not. 


Evidence Line 4: Laboratory Studies 


The fourth line of research involves using this iterative learning paradigm 
with real live people. Kirby and colleagues have done a number of these 
studies in the laboratory. Kirby has replicated the computer simulation 
results with humans, and he has shown that change can happen over far 
fewer trials. 
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In a study with Erica Cartmill, an expert on hand 
gestures, human subjects were asked to learn 
and teach the meanings of hand gestures 
that were paired with dynamic scenes ona 
video, like a ball rolling or bouncing down 

an incline. 


Among early generations of learners, the 
gestures that people produced were extremely 
idiosyncratic. After only 10 iterations of learning, 
the form and order of gestures all became much more 

uniform, both within and across subjects. It seems that just like virtual 
learners, real learners impose structure to make the learning task easier 
and more systematic over time. 


Taken together, these four lines of work suggest that brains do not 
necessarily need to change to make a tight fit with language; the 
very act of learning itself may exert a powerful pressure that turns 
unstructured communication into structured language. 


The Nativist Defense 


Most experts now accept the anti-nativist and cultural-learning 
mechanism for language evolution. However, accepting this does not 
necessarily mean rejecting the nativist, biological-adaptation account 
entirely. Both mechanisms may be right on different timeframes. 


Even if communication originally adapted itself to the brain, it did 
eventually evolve into what we now call language. And once it became 
language, what was to stop it from exerting its powerful influence little 
by little back on the brain? Brains do change over evolution; they just 
change more slowly than language. 


There is good reason to believe that brains do adapt to language. For 
example, if our ancestor’s brains became used to using a tool with 
such incredible power, why wouldn’t their brains eventually change to 
become even more receptive to language? 


6. HOW THE BRAIN CREATED LANGUAGE 42 


When the brain has grown so accustomed to using language to get 
things done, it becomes a default way to pursue goals. When we so 
regularly and effectively achieve those goals with language, this feeds 
back on the brain: The more useful language is, the more the brain 
adapts to become receptive to it. 


Suggested Reading 
Christiansen and Chater, Creating Language. 
Dennett, From Bacteria to Bach and Back. 


Sandler, Meir, Padden, and Aronoff, “The Emergence of Grammar.” 


Questions to Consider 


Can you think of examples of cultural memes that live outside our 
heads but are part of our minds? For example, the philosopher 
Daniel Dennett argues that religious stories are one such meme. Do 
you agree with him? 
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7° 

GESTURE AND THE 
ORIGINS OF HUMAN 
LANGUAGE 


This lecture looks at how different timeframes 
converge across different levels of analysis to 
tell a story of what our earliest attempts at 
language might have looked like. The lecture 
does so by highlighting an everyday behavior 
that often flies under the radar: hand gestures. 


Additionally, this lecture presents the claim that 
the human body—particularly the hands—was 
the foundation on which humans initially built 
language. To support this claim, this lecture 
puts together four different pieces of a puzzle. 


The first piece comes from neuroscience: 
Spoken language and manual gestures are 
bound in the brain. The second piece stems 
from gesture’s present-day prevalence: It 

is everywhere. The third considers how 
humans use gestures uniquely compared to 
other species. The fourth concerns the many 
positive functions of gesture for present- 
day communication. 


Neuroscience 


Most scholars now believe that language is not a premade and 
specialized module in the brain at birth; instead, they see it as emerging 
from more general cognitive and social skills combined with physical 
interactions with others over development. 


Gestures might have functioned as an early precursor to language. One 
basic function of a gesture is to represent things not in the here and 
now. For example, our ancestors could point to places out of sight (over 
a hill or in a cave) or mime building a fire by flinting rocks that weren’t 
actually present. 


However, direct archaeological support for this claim is difficult to 
find. Unlike the origin of actual physical tools, there are no fossils for 
language and gesture. Because of this, one approach is to study the 
brain structures of present-day humans to reveal clues about the past. 


Human language is built on brain regions that contribute to controlling 
the hands. For example, structures in the left hemisphere involved in 
language are also involved in grasping objects. 


The overlap between language and action in the brain is not just 
general; it is highly specific. We know from the pioneering work of 
neurosurgeon Wilder Penfield that the human brain is organized 

in a topographical way, like a city with distinct but interconnected 
neighborhoods. Additionally, Friedemann Pulvermiller of the Free 
University in Berlin showed that language was also mapped onto motor 
areas of the brain. 


Work like this has led to a theory of brain functioning called simulation 
theory, which states that one way we understand language (and 
concepts more generally) is by neurally simulating how we would 
actually interact with real objects in the world. Language uses the body 
just as much as the body uses language. 
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Prevalence 


If gesture is still linked to speech in our current language use, this would 
suggest an even tighter relationship in our past. Research shows that 
people of all ages, languages, and cultures gesture when they speak. In 
fact, children gesture before they speak. 


Gesture accompanies the nearly 7,000 distinct spoken languages 
documented in the world. It is not new to the scene: Gesture has been 
depicted in religious art for millennia in all parts of the world. 


David McNeill, a researcher at the University of Chicago, has the 
most established theory for why we gesture. McNeill argues that the 
reason we gesture when we speak is that gesture and speech are part 
and parcel of language. They are actually two different—but tightly 
coupled—incarnations of our thoughts. 


Because speech is highly arbitrary and 
conventional, words are not inherently 
related to their meanings. On the 
other hand, gesture is imagistic and 
idiosyncratic, which means that the 
form of a gesture directly relates to 

its meaning. McNeill’s theory states 
that gesture and speech are simply 


Blind Gestures 


Blind people use 
gestures, including 


when they are 
two sides of the same thought, and talking to other 
only by considering them together is blind people. 
it possible to understand someone’s 
complete message. 


Uniquely Human Usage 


If gestures are so prevalent and such a fundamental part of language in 
humans, we should use them differently than animals that do not have 
language. The leading expert on comparing gestures in humans and 
nonhumans is Michael Tomasello. 
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Tomasello theorizes that human language is unique not because it is 
special in and of itself. Instead, it is special because it is built on special 
social skills that have evolved distinctively in humans. 


People are driven to share attention and cooperate with one another. 
People are also remarkable mind readers. For Tomasello, both of these 
basic social functions manifest themselves first in children’s hand 
gestures. Even though animals, such as chimps, can be trained to point 
at things, their gestures all attempt to satisfy immediate desires, such as 
obtaining a reward. Human gestures have two additional functions. 


One is simply to share attention with other people. To appreciate this, 
consider how much pointing happens when a person reads a picture 
book to a young child. The child is not requesting anything: If she points 
to a picture of the moon, she does not actually want the moon. She 
simply wants to share attention to it. 


A second unique function is to offer help. Imagine a person in a café 
seeing another customer drop a $10 bill as she walks by. Most people 
would reflexively make eye contact with the person and point to the 
money on the ground, even though they would profit from taking the 
money for themselves. This cooperative use of gesture is distinctive 
to humans. 


In both of these examples, note the complex mind-reading that is 
required. In the first case, the adult and the child each have to know 
that pointing to the picture of the moon is not a request for the moon, 
and crucially, they have to know that the other knows it too. In the 
café example, the other customer has to know that the pointer is not 
requesting the $10 bill for themselves. 


According to Tomasello, these differences in how humans and 
nonhumans use and interpret gestures are at the heart of what makes 
us different from all other species. Humans use gesture as a way to read 
minds and coordinate actions with others. This mind-reading and social 
cooperation are what language is all about, and the fact that gesture is 
such a handy tool for doing it speaks volumes about its importance. 
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The External Functions of Gesture 


People do gesture for others, but that’s only half the story. Scientific 
research shows that we also gesture for ourselves. These can be 
thought of as the external function—we gesture for the benefit of 
others—and the internal function—we gesture for our own benefit. 
A single gesture often has both functions. 


First, consider what gesture does for other people. Outside of sign 
language, the clearest example of a gesture designed for other people is 
an emblem. Emblems are conventionalized hand shapes that often stand 
alone in their meaning. The OK sign is an emblem. 
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According to Adam Kendon, one of the pioneers of modern gesture 
research, emblems have the advantage of not needing speech, which 
works well in situations requiring discretion or stealth. Another advantage 
is that they are effective from a distance in noisy environments. 


A second function is that gestures help make a message clearer. 
Additionally, because gesture adds meaning to speech, it can be used 

to help people learn new things. Co-speech gestures also help a person 
make a better impression. For example, in the context of the classroom, 
research shows that students like teachers who gesture during a lecture 
more than ones who do not. Students also say that they learn more 
from teachers who gesture. 


Gestures can also function to build rapport between people. As many 
successful salespeople and politicians already know, subtly mimicking 
another person’s gestures can increase persuasiveness. 


The Internal Functions of Gesture 


Embodied theories of language not only explain why we gesture; they 
actually predict it. Gesturing makes speaking much easier. Moving your 
hands can move your mind. An extreme example of this is what Susan 
Goldin-Meadow calls gesture-speech mismatches. A mismatch is when 
someone Says one thing with their words but another with their hands. 
Goldin-Meadow shows that only the combination of both things reveals 
the complete thought. 


For example, adults know that changing the appearance of a tall glass 
of water by transferring it to a short, wide dish does not change the 
essence of that thing. The amount of water has not changed, only the 
appearance. However, a seven-year-old would likely be fooled by the 
change of appearance. 


Just before seven-year-olds figure this out, their gestures often lead the 
way. Goldin-Meadow’s research shows that children who are just at the 
transition of not being fooled by appearances produce gestures that 
complement their speech: If they speak about the different heights of 
the containers, they simultaneously gesture about the different widths. 


7. GESTURE AND THE ORIGINS OF HUMAN LANGUAGE 


49 


In this way, the information in the gesture compensates for the 

content of the speech and reveals a more sophisticated understanding: 
The containers may be different heights, but that is offset by their 
different widths. This basic phenomenon applies to any problem where 
there is a significant transition in knowledge, and it occurs in children as 
well as adults. 


Gesture reflects peoples’ changing knowledge, but it goes beyond that: 
It also changes their knowledge. For example, allowing someone to 
gesture on a hard math problem can give new insights into how to solve 
that problem. On the other hand, prohibiting someone from gesturing 
disrupts their ability to think. 


Suggested Reading 
Church, Breckinridge, Alibali, and Kelly, eds., Why Gesture? 


Kelly, McDevitt, and Esch, “Brief Training with Co-Speech Gesture 
Lends a Hand to Word Learning in a Foreign Language.” 


Tomasello, Origins of Human Communication. 


Questions to Consider 


If you think about the five different components of language, which 
ones lend themselves more—and which ones less—to co-speech 
hand gestures? What aspects of language may have started in the 
body and then were liberated from it over evolution? 
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8. 
DEVELOPMENT: 
A MIND UNDER 
CONSTRUCTION 


This lecture discusses three big 
mechanisms for development: 
innate knowledge, powerful 
learning tools, and unconscious 
tuition from others. These 
mechanisms interact over 
development to create a 
dedicated face-processing region 
in the brain. 


Chuck Close and the FFA 


The artist Chuck Close paints giant, abstract portraits that are made up 
of hundreds of little boxes on a grid that—up close—look like a jumble 
of pixelated colors. However, when viewers stand back at a distance, the 
pixels come together to create the distinct image of a face. The part of 
the brain that flips this switch from confusion to clarity has been called 
the fusiform face area, or FFA for short. 


This lecture’s guiding question involves the FFA: Is the FFA designed 
to process faces specifically, or could it have served a more general 
function, like processing any type of meaningful visual information? 


Development 


To introduce the scientific study of human development, it is necessary 
to have a philosophical foundation for the big issues. The biggest issue 
concerns the age-old question of nature and nurture. Historically, these 
have been treated as black or white options: Either people are born 
with what they need to know about the world, or they need to learn it 
from experience. 


These extremes have carried a legacy into modern-day linguistics, 
neuroscience, and psychology, but they have given way to a more 
balanced view. It is now commonly accepted that the human mind is the 
result of both nature and nurture. 


The most exciting questions now in developmental psychology and 
neuroscience are concerned with determining exactly what is built 
into humans at birth, and how and when those things interact with 
the environment throughout development. 


A helpful guide in addressing these questions is the book The Scientist 
in the Crib, written by Alison Gopnik, Andrew Meltzoff, and Patricia 
Kuhl of the University of Washington. The book takes an approach 
similar to this course’s 3-D framework, and it outlines three major 
mechanisms that drive development. 
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Those mechanisms are innate knowledge, powerful learning tools, 
and unconscious tuition from others. The first and the third best 
correspond to the classic nature-nurture dichotomy, and the middle 
one serves as a bridge between those two. 


Innate Knowledge 


When developmental psychologists speak of innate knowledge, they 
are talking about the baggage that humans have accumulated over 
the course of evolution that they bring to personal development. 
There is vigorous debate about the nature of this knowledge. 


On one hand, there is the domain-general view. Domain-general 
knowledge refers to built-in abilities that are very broad and can be 
applied to many different domains. The other type of innate knowledge 
is domain-specific knowledge, which is something that is innately 
specified for a very particular function. 


This idea brings up the guiding question about the FFA: Is it designed 
to process faces specifically, or does it serve a more general function? 
Most researchers now believe that the FFA serves a more general 
function at birth. Evidence for this comes from clues on multiple levels 
of analysis and different timeframes. Here are two clues for now: 


First, although research does show that babies have a clear preference 
for faces over other objects at a very early age, findings suggest that 

a more general mechanism, such as symmetry detection, might drive 
our early preference for faces. Additionally, babies also look at animal 
faces for longer than objects, which points away from a specific 
preference for human faces. 


The second piece of evidence comes from neuroimaging studies on 
infants. Rebecca Saxe and her team at MIT investigated brain activation 
of babies as young as four months old when viewing images of faces 
versus natural scenes. Although there were large-scale brain areas that 
differentiated faces and scenes, it was not localized in the FFA, like it is 
in adults. 
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Powerful Learning Tools 


It seems that the brain must undergo some significant changes in order 
for the FFA to become specialized for processing faces. This idea leads 
to the topic of powerful learning tools. Even with innate knowledge— 
specific or general—we also need some inborn cognitive and social skills 
to help us build on that knowledge. 


By far the most influential pioneer of innate learning 
mechanisms was the Swiss developmental 
psychologist Jean Piaget. The ability to make 
associations and experience emotions are 
necessary innate tools. According to Piaget, 

children also possessed much more sophisticated 
tools beyond those. 


For example, Piaget argued that children are 

innately driven by curiosity, and they are predisposed 
to a type of systematic thought and action that would 

be familiar to any laboratory scientist. Just like scientists, babies are 
innately capable of making predictions about the world, manipulating 
things in it, and observing and recording the outcomes. This is why 
Gopnik, Meltzoff, and Kuhl referred to babies as “scientists in the crib.” 


Much of babies’ scientific exploration is done through direct 
experience—that is, learning by doing. Piaget also stressed another 
powerful learning mechanism: learning by watching. Imitation is one of 
the most powerful tools that humans have for learning about the world. 


Plasticity 


The grandmother of all mechanisms that unites all learning and 
development is the neural process of plasticity—that is, the property 
of the brain that enables changes to its neural connectivity. The idea is 
fairly simple: If a particular neural connection produces useful behavior, 
that connection is strengthened. If it is very useful, it might even sprout 
new connections to strengthen it further. However, if a connection no 
longer serves a useful function, it is weakened or even eliminated. 
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Our brains are much more plastic 
early in life versus later in life. Plasticity 
functions to help us specialize during 
development. Humans come into 

the world with an empty survival kit 
packed into our DNA. We have to learn 
to survive. This is in contrast to many 
other species that enter the world 
ready to go. 


We use plasticity to learn what is 
important for our survival. Because 
humans inhabit such a complex physical 
and social world, it is not possible to 
genetically build in everything we will 
need to survive. 


The fact that adults have such a clear 
neural architecture for processing 
faces—and infants do not—suggests 
that plasticity has sculpted our brains 
during development to make it this way. 
This reorganization comes with some 
real benefits: Because faces convey 
such important social information, 
having a fully dedicated brain region to 
quickly and accurately process them 
would serve a real survival function. 


This clarifies the tradeoff between 
being more plastic early in life and 

less plastic later in life: Being open to 
change early on helps the brain learn 
and commit to what’s important, and 
locking in later allows it to process that 
information much more efficiently. 
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This raises a big question: What causes the brain to lose its plasticity 
and become more rigid and specialized? One possibility is that it is just 
an innate mechanism. This is true to some extent, but scientists believe 
that a more powerful force is responsible. This is Gopnik, Meltzoff, and 
Kuhl’s third and final mechanism of development: unconscious tuition 
from others. 


Unconscious Tuition from Others 


Unconscious tuition from others has to do with a sort of guidance, 

but not in the traditional sense. Explicit guidance can powerfully shape 
the mind; for instance, that is why we have schools. Unlike its formal 
counterpart, unconscious tuition is done spontaneously, naturally, 

and inadvertently by parents and other adults, older siblings, and peers. 


Children are prolific learners, and they start their informal education 
from the day they are born. Other humans bombard them with social 
aspects of culture, including language. Human babies are awash in 
words from the moment they are born. 


For the first three years of life, most children hear on average more 
than 10,000 words a day. Some who are raised in highly verbal 
environments hear more than 30,000 words a day. Most of these 
words are not even directed at the child. They are just a natural part of 
the environment. 


The mechanism of unconscious tuition can be applied to the example 
of face processing. Informal face training for humans starts on the first 
day. It begins when parents hold newborn babies about a foot from 
their own face. It continues nonstop from there. 


Our brains change to more efficiently process faces with every new 
exposure. Capitalizing on the brain’s plasticity, we dedicate a chunk of 
cortex specifically for faces. Real estate is expensive in the brain, so this 
is a big commitment. 


Conclusion 


Even though the FFA becomes specialized for faces, does that 
necessarily mean that it is innately designed for faces specifically? This 
was the question asked in the late 1990s by Isabel Gauthier, a cognitive 
neuroscientist. In a study, Gauthier used fMRI to investigate FFA activation 
in adults who were and were not bird experts. She reasoned that bird 


experts have comparable experience identifying birds as they do humans. 


If the FFA is innately designed specifically for human faces, this bird 
expertise should not matter, and human faces should produce more 
FFA activation than faces of birds. If, however, the FFA is driven by visual 
expertise more generally, bird experts should have similar FFA activation 
to human faces AND bird faces. 


Whereas non-bird experts showed the signature FFA activation only for 
human faces, bird experts showed comparable activation for both bird 
AND human faces. This activation was related to their behavior: The 
more active the FFA was for birds, the better the experts remembered 
their faces in a later memory test. This effect has been replicated many 
times, and it has even been generalized to other areas of expertise, such 
as images of cars and chess positions. 
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Findings such as these have caused most of the scientific community 

to stop using the term fusiform face area. Many now instead use the 
term fusiform expertise area. The idea is that humans are born with 

a part of the brain that has the domain-general ability to dedicate itself 
to efficiently processing highly salient and important visual information, 
be it faces, birds, cars, or whatever else is important. If someone is 
exposed to enough of this visual input, it triggers the fusiform expertise 
area to undergo plasticity and become specialized. 


Suggested Reading 


Gauthier, Tarr, Anderson, Skudlarski, and Gore, “Activation of the 
Middle Fusiform ‘Face Area’ Increases with Expertise in Recognizing 
Novel Objects.” 


Gopnik, Meltzoff, and Kuhl, The Scientist in the Crib. 


Questions to Consider 


Gopnik, Meltzoff, and Kuhl refer to babies as “scientists in the 

crib.” Can you think of other examples beyond language where 
developing humans might be actively testing hypotheses about 
their world? For example, how might you view the so-called terrible 
twos or even teenage rebellion from this perspective? 
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9. 
SPECIALIZING 
IN SPEECH SOUNDS 


There are almost 7,000 languages on 
the planet, and they are composed 
of hundreds of different phonemes. 
Faced with such a diversity of speech 
sounds, it’s not unreasonable to think 
that babies come into the world with 
very limited auditory abilities, and 
then, through massive exposure and 
experience, they start to hear the 
sounds of their particular language. 
However, that is not how evolution 
has solved the problem. 


Phonetic Abilities 


Rather than coming in with blank slates, babies are evolutionarily 
designed to enter the world with some very impressive phonetic 
abilities. We have learned this from a groundbreaking line of work by 
several researchers on infant speech development. 


The pioneering paper was published in 1971 by the speech scientists 
Peter Eimas and Paul Jusczyk at Brown University. Eimas and 
Jusczyk were interested in whether babies could process phonemes 
categorically, like adults do. 


For example, Japanese adults know that the short /u/ phoneme in shujin 
belongs to the category of short vowels, but the long /uu/ phoneme 

in shuujin belongs to the category of long vowels. This categorical 
distinction between vowel length doesn’t exist in English: Elongating 
vowels in English may sometimes add a nuanced connotation to words, 
but it does not qualitatively and invariably change their meanings as it 
does in Japanese. 


A challenge for the researchers was that it is 
impossible to directly ask young babies how 
they hear different phonemes categorically 
or continuously. Solving this problem was 
the creative insight of Jusczyk and Eimas. 


They took advantage of something that 
babies are born to do: suck. All babies 

have an innate reflex to suck more when 
they are interested and less when they 

are not. The idea was to present babies a 
phoneme from one category over and over 
until they got bored and began sucking less, and 

then measure what happened when the babies heard a 

new phoneme from a different category. Remarkably, Jusczyk and Eimas 
found that even babies as young as one month old could categorically 
distinguish the sound /ba/ from /pa/. 
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In another study, Janet Werker and Richard Tees conditioned six-month- 
old English-speaking infants to turn their heads when they heard a 
difference between two slightly contrasting phonemes. Babies could 

do this in English, and they could do it in Hindi as well. These babies had 
never been exposed to a single Hindi contrast in their entire lives, so 
this ability was not learned from linguistic experience. Instead, it must 
be innate. This finding has been replicated with dozens of languages 

and with younger babies, so this is a very powerful and robust form of 
innate knowledge. 


Neural Mechanisms 


Neural mechanisms underlie this phoneme specialization process. 
An important concept here is that of topographical organization. 
Different lobes of the brain have different functions; along the same 
lines, there are smaller regions within those lobes that have more 
specific functions. 
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For example, at the outer edges 

of the occipital lobe, the fusiform 
expertise area is specialized for 
processing complex visual objects like 
faces. This is in contrast to different 
parts of the visual system in the occipital 
lobe that are specialized for other complex 
objects, like furniture, plants, and even words. 


Outer 
occipital lobe 


Occipital lobe 


auditory The same thing is true for auditory processing. 
Nair The primary auditory cortex (also called A1) 

is located in the superior temporal lobe along 

a stretch of cortex called Heschl’s gyrus. After 

a sound reaches the ears, A1 is the first part of the 
neocortex that processes these sounds. The cells in 
Ai that process these sounds are highly organized. 


Phonemes are processed in a network extending 
beyond A1. Scientists believe that there are dedicated 

and specialized cell networks for individual features of the 
phonemes in one’s native language. The function of having specialized 
cells for aspects of native speech sounds is to help humans process 
their language quickly and accurately. 


An important feature here is neural plasticity—that is, the brain’s 
incredible ability to change itself. This is the quintessential domain- 
general mechanism because it applies to everything. The brain starts 
with some innate structure, but through experience, plasticity allows 
those structures to become more specialized and efficient. 


Synaptogenesis and Synaptic Pruning 


Two complementary aspects of plasticity are synaptogenesis and 
synaptic pruning. Synaptogenesis refers to the process of the brain 
creating an increasing number of dendrites to make new synaptic 
connections. Synaptic pruning refers to the process of the brain 
cutting back on these synaptic connections. 
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Research by the neuroscientist Peter Huttenlocher at the University of 
Chicago has shown that over the first few months of a human baby’s 
life, A1 undergoes a rapid period of neurogenesis, but after about six 
months, it undergoes neural pruning. 


It is now believed that babies are born with neural connections that 
allow them to immediately start paying attention to whatever sounds 
they may hear in their particular language environment. Over the first 
six months of life, babies start to expand the neural connections that are 
involved in processing native speech sounds. After that, the brain starts 
to prune the connections that are not involved in processing native 
speech sounds while strengthening the ones that are. 


The end result after one year of life is a reorganized auditory 

system that has roughly the same number of synaptic connections 

in Ai. However, the nature of those connections has been radically 
transformed. At first, the connections allowed the baby to process all 
speech sounds fairly well, but after one year, the neural reorganization 
allows the infant to process native speech sounds extremely well. 


The Cost 


There is a cost to this plasticity. It means that the cells that used to 
process phonemes in other languages are gone. These connections 
have been pruned away because they served no useful purpose for the 
developing baby. 


All this has some significant consequences for how our brains hear 
speech sounds. Pat Kuhl is one of the leading experts on neural aspects 
of phoneme development, and she describes a fascinating function of all 
this neural reorganization: the perceptual magnet effect. 


When the brain commits to certain phonemes in one’s native language, 
it creates prototypal categories of each of those phonemes. These 
prototypes are useful because every speaker of a given native language 
produces slight variations in their phonemes. This is true even for 
people speaking the same dialect of English. 
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If the variations are too great, the magnet can’t quite reach the word, 
and the result will be confusion. However, as long as the phonetic 
variations are not too great, the perceptual magnet constantly 

bends what it hears to assimilate variations into the prototypes. In 
other words, most of the time, the perceptual magnet unconsciously 
translates those differences into the sounds that make the most sense 
to a person’s brain. 


Brains and Distortion 


Being exposed to one’s native language changes a person’s brain to 
warp what it actually hears in the world. That means that we don’t have 
objective access to auditory information in the environment. Everything 
is distorted. 


From the perspective of this course’s 3-D framework, an important 
question is what possible function this could serve. One possibility 
is that it helps our survival. Donald Hoffman, a cognitive scientist, 
argues that evolution has designed humans to massively distort 
our perceptions. 


He argues that the function of perception (and cognition more 
generally) is not to accurately process the world around us. Rather, the 
function is to help us better interact with that world. Hoffman posits 
that millions of years of evolution have built brains that distort reality 
and allow us to live in an illusion that helps us function better. 


Returning to the concept of the perceptual magnet, it makes sense 
that our brains would gloss over variations of phonemes if it ultimately 
allows us to understand a wider range of speakers in a faster and more 
accurate way. The goal is not to hear every variation of what a speaker 
says; the goal is to accurately identify what a speaker means with a 
spoken message. That is the problem that evolution has tried to solve, 
so it makes sense that we would adapt our brains to do it better. 
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This distortion is not without drawbacks. One consequence of having 
a strong perceptual magnet for a person’s native language is that it 
becomes very hard to relearn how to hear sounds in a foreign language. 


Suggested Reading 
Hoffman, The Case against Reality. 
Kuhl, “Early Language Acquisition.” 


Werker and Tees, “Cross-Language Speech Perception.” 


Question to Consider 


Synaptogenesis in the primary auditory cortex (A1) peaks at around 
six months of age, and then the process of synaptic pruning begins. 
Consider this fact: The prefrontal cortex undergoes synaptogenesis 
throughout adolescence and early adulthood, and only then does it 
begin to slowly prune. What is the function of these two different 
neural timetables? 
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10. 
NAVIGATING A WORLD 
OF WORDS 


There has been a great deal of research empirically 
investigating how young children map meanings onto 
words in real-life language development. As this lecture 
shows, human babies possess some innate and learned 
heuristics that give them a head start. 


Patterns and Word Boundaries 


From ages 6 to 12 months, babies start to narrow their perceptual focus 
on native language phonemes. This age range is also when they begin to 
understand words in their native language. 


In 1996, Jenny Saffran published a paper that changed the way we 
think about how infants learn language. She found that eight-month- 
old babies are able to extract statistical regularities in continuous 
speech to identify words and word boundaries—that is, when one 
word ends and another begins. 


Babies are such good pattern recognizers that they can learn what 

is a word—and what is not a word—from extremely limited auditory 
input. This is a very robust finding that has been replicated many 
times. Additionally, it has been extended in interesting ways, including 
these three: 


1. The ability appears to be innate. A Finnish team of researchers 
has measured electrical activity of newborn baby brains and 
found that they also use statistical regularities of syllable 
combinations to identify words and word boundaries. 


2. Research by Saffran’s advisors, Richard Aslin and Elissa 
Newport, has shown the ability to detect patterns in speech 
probabilities is not unique to humans. Monkeys can do it too. 
This suggests that the skill is built on mechanisms shared with 
other species. 


3. This pattern-recognition ability shows up in other modalities, 
such as vision, suggesting that it is a domain-general skill and 
not one specifically designed for language. 


The takeaway point is a familiar one: Humans are born with some innate 
domain-general knowledge and powerful learning tools that allow them 
to get an early jump on learning language. 
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Identifying Meaning 


Once a baby has identified a word, the baby needs to learn what 

the word means. Babies receive help from two places: Adults provide 
useful hints to clear up ambiguities, and babies have some powerful 
biases and heuristics that help constrain potential meanings. 


Ellen Markman was one of the first developmental psychologists 
to systematically study the cognitive constraints on children’s early 
word learning. She identified three in particular that greatly help 
children: the whole object assumption, the mutual exclusivity bias, 
and the taxonomic assumption. 


The whole object assumption refers to the tendency of a language 
learner to think that a word refers to a whole object and not its parts. 
This bias can cause children to make occasional mistakes, but overall, 
it is very helpful. 


The mutual exclusivity bias is one of the most robust heuristics that 
people use in understanding language. Essentially, this bias keeps 
people from assigning new words to objects they already know. 
(Note that in practice, items often have 

multiple labels, however.) 


Markman’s third constraint is 
the taxonomic assumption. Once 
children move past mutual exclusivity 
and have accepted that objects 
often have multiple labels, they 
start to appreciate that words 
often refer to categories of 
things. For example, if a person 
were to point to a dog, she 

could label it as a dog or more 
generally as a pet or animal. 
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Category Labels 


Researchers have explored how early babies treat new words as 
category labels. Babies are surprisingly savvy at very young ages. 
In one remarkable study, Alissa Ferry, Susan Hespos, and Sandra 
Waxman tested categorization at the raw age of three months. 


They presented these infants with pictures of different species of fish. 
With each picture, they also played an audio track that said: “Look at 
the toma. Do you see the toma?” They did this eight times with eight 
different pictures of fish, but they used the same audio track for each 
one. Next was the test phase: The babies were shown two pictures. 
One was a new exemplar of the fish category, and one was a picture of 
a totally new category: a dinosaur. 


The key measurement was to record which objects the babies looked 
at more. Remarkably, these preverbal infants differentiated the two 
pictures, looking longer at the image within the same category, the fish, 
than the one in the new category, the dinosaur. 


A relevant question here is: Does this really have to do with babies 
using speech to form categories, or does it just indicate that the 
babies had gotten into the habit of looking at fish, not dinosaurs? 
The researchers addressed this valid question by repeating the exact 
same study with a different set of three-month-olds, but there was 
one change: They replaced the auditory naming of the objects with 
an auditory tone during the exposure phase. There was a sound that 
accompanied each picture, but no speech. 


After eight exposures of fish and tones, the babies were shown the two 
objects: a new fish and a dinosaur. The babies did not differentiate the 
two pictures. It was as if they had not been forming a category at all 
without a repeated verbal label. 


It’s possible that the babies didn’t really understand what the words 
meant, and they just learned that when they hear language and see 
similar things, they should start grouping those things in their minds. 
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However, these findings are still impressive. They mean that very 
young children might be using language to help orient their minds to 
categorize things in the world, and this could help them constrain their 
early guesses about what words actually mean. 


Humans are not the only ones who use language to categorize the 
world. There is evidence that dogs can learn words in a categorical way. 


Suggested Reading 


Markman, “Constraints Children Place on Word Meanings.” 


Saffran, Aslin, and Newport, “Statistical Learning by 8-Month-Old 
Infants.” 


Questions to Consider 


The film Arrival is about a linguist trying to decipher the language 
of an alien species that has landed on earth. Watch that film and 
consider Quine’s conundrum. Given that aliens may differ from 
humans in every way—biologically, psychologically, and socially— 
would it ever be possible to truly understand an extraterrestrial 
mind? For that matter, will humans ever truly understand other 
terrestrial minds? If this intrigues you, check out the philosopher 
Thomas Nagel’s provocative essay called “What’s It Like to Be a 
Bat?” When you finish, try to answer his question. 
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11. 
LEARNING TO PLAY THE 
GAME OF LANGUAGE 


The pragmatics of language is very important. Without 
understanding the richness of social intentions, 
motivations, and context, there is no hope for people to 
have successful communication. This lecture discusses 
what young children know and learn about their social 
world and how they use that information to navigate the 
endless ambiguities of language. It also looks at some 
interesting studies related to language and the brain. 


Joint Attention 


In the 1970s, the psychologist Jerome Bruner performed some classic 
research. Using cutting-edge technology of his time, Bruner analyzed 
videotapes of nine-month-olds spontaneously interacting with 
caregivers in their homes. He was looking for social behaviors that could 
be signs that babies were learning about the goals, motivations, and 
intentions of adults. 


Bruner made many interesting observations, but the one that best links 
up with language learning is a behavior he dubbed joint attention. This 
referred to how babies seemed especially interested in sharing visual 
focus on objects with adults. Bruner argued that when babies and adults 
are looking at the same thing, they mutually know that they’re both 
looking at that thing. 


Given that joint attention emerges just before children start speaking, 
it likely serves as a mechanism for early word learning. Here is how 

it might work: When an infant and adult mutually know that they are 
looking at the same object, the infant infers that any word that the 
adult says must refer to the thing that they are both looking at. It is as if 
the child and adult have a tacit agreement that labeling objects always 
happens in the social context of joint attention. 


Evidence of Joint Attention 


Because infants hear thousands of words every day, having a social 
mechanism that links some of those words to particular objects in the 
environment greatly simplifies the learning task. However, this raises 

a question: What is the evidence that infants use joint attention to 
actually learn words? 


In one of the first systematic attempts to explore this question, Dare 
Baldwin, a developmental psychologist, did a series of studies on 
children in the throes of early word learning. In one study, Baldwin was 
interested in whether 18-month-olds would learn a name for a toy if 
there was no joint attention with an adult. 
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The children were placed in a room with a few novel toys in it. Crucially, 
these were toys that they had never seen, so there is no way children 
could have known the names for them. When they started playing 

with the toy that interested them, an experimenter labeled it by saying, 
“Dawnoo. There’s a dawnoo.” 


This labeling occurred in two different conditions. In one condition, 

the experimenter was right next to the child looking directly at the toy. 
Inthe other condition, the experimenter was next to the child but was 
sitting on the other side of an opaque screen. The child couldn’t see 
where the experimenter was looking. In both cases, the child heard the 
exact same speech—the only difference was that in one case, the child 
and adult shared joint attention, and in the other, they did not. The main 
finding was that when there was joint attention, children frequently 
mapped the word dawnoo onto the toy they were playing with. When 


there wasn’t joint attention, children did not reliably learn the new word. 


Lending a Hand 


Eye gaze is not the only useful nonverbal tool in the word learning 
process. Hand gestures also play a huge role. Infants and adults 
communicate with hand gestures well before they start communicating 
with words. Pointing gestures, in particular, are especially prominent. 


Babies point to get things they want and to comment on things 

and inform people about them. For example, Ulf Liszkowski and his 
colleagues have done a series of experiments with one-year-olds 
showing that they produce gestures to serve both of these functions— 
to comment and to inform. 


It is notable that much of this early nonverbal communication happens 
before infants even begin to speak. This implies highly sophisticated 
perspective-taking skills for preverbal infants. For example, pointing to 
interesting objects suggests that infants believe that other people will 
also find the same things interesting. 
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The flip side is also likely: Paying attention to where adults point helps 
an infant understand what is interesting to other people. All of this 
early exchange of information helps a baby understand what sorts of 
things might be interesting to others, and this could be very useful for 
understanding what words refer to. 


Taking Perspective 


Perspective taking is the key to the savvy pragmatic use of language. 
Children have some decent perspective-taking abilities even within 
their first year of life, but as they head into their second year, they go 
to the next level. 


This period marks the start of a gradual transition from having a 
somewhat egocentric view of the world toward one that carefully 
considers the different perspectives of others. There is even a name for 
this skill: theory of mind, or ToM for short. 


This gradual development of perspective-taking skills is highly 
correlated with language development. As perspective taking becomes 
more sophisticated, so does language. This has led some to hypothesize 
that perspective taking is a key mechanism for language development. 
That certainly seems to be the case for things like joint attention and 
early word learning, and it is possible that improvements in theory of 
mind help with later stages of language development as well. 


Derailment 


One way to approach this link between perspective taking and language 
acquisition is to consider cases in which perspective taking goes off the 
tracks in development. An example is autism spectrum disorder (ASD). 
Children with ASD exhibit significant language impairments, and they 
also have difficulty taking the perspective of others. 


Additionally, they gesture differently. Developing children will typically 
point to things to request them. These are called imperative gestures. 
They also gesture to comment and to inform. In contrast, the gestures 
of infants and toddlers with ASD are almost always imperative. 


Eye gaze is also different. Most children with ASD fail to engage in joint 
attention, and they often don’t make eye contact with people in the 
same way. In one eye-tracking study, Warren Jones and Ami Klin showed 
that for the first two months of life, children at risk for autism found 
eyes equally interesting as typically developing children—they looked 
just as long at them. However, from ages two to six months of age, the 
group at risk of ASD looked less and less toward eyes compared to the 
control group. 


All children are born with an innate preference for looking at human 
eyes. Most children quickly realize that the eyes offer social clues about 
people’s interests and intentions. This realization, in turn, causes even 
more attention to the eyes. From there, the cycle repeats itself, and 
attending to the eyes eventually becomes a habit by the time children 
start learning language. 
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Things are different for children with ASD. Although they start life just 
as interested in human eyes, for some reason, they never learn that eyes 
are a gold mine of social information. They never develop the habit of 
attending to the eyes during language learning. 


The Empathy Switch 


The condition of psychopathy also presents intriguing links between 
perspective taking and language. The popular stereotype of a 
psychopath is a cunning person with no empathy, but research suggests 
that psychopaths may be more empathic than previously thought. 


In a 2013 study, Christian Keysers and colleagues teamed up with Dutch 
forensic psychiatric clinics to measure empathic brain responses of 
prisoners diagnosed with psychopathy. Under very close guard, the 
team had inmates come into the lab to receive fMRI scans of their brains 
while they viewed pictures of people in pain. 


In non-psychopath control subjects, these images strongly activate 
many of the same parts of the brain that are active when people are 
actually in physical pain. This is interpreted as a type of neural empathy: 
People understand the pain of others by simulating it in their brains. 


Psychopaths are different. When the inmates were in the scanner, 
they showed much less empathic activation in these pain regions, 
suggesting that they had difficulty taking the perspective of the people 
in the pictures. 


When the inmates were explicitly asked to take the perspective of 

the people in pain, their results looked much more like the control 
group’s. However, there were some big differences in the frontal lobes, 
suggesting that psychopaths were engaged in a much more strategic 
and controlled form of thinking. 


This finding adds nuance to the traditional view of psychopathy: It’s 
not that psychopaths are incapable of taking the perspective of others. 
Rather, they are simply very strategic about when they do it. Empathy 
is a default mode for most people, but it appears to be optional 

for psychopaths. 
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The study suggests that one reason why psychopaths can be such 
smooth talkers is that they have an empathy switch: When it serves 
their purposes, they can turn it on and see things from another person’s 
perspective. After all, the best con artists are masters at telling people 
what they want to hear. 


Matters change abruptly after the charm has achieved its goal. Once 

they have hooked and drawn someone in, having empathy may get in 
the way of doing bad things to a victim, so it would be highly adaptive 
for psychopaths to turn it off. This is the worst of both worlds: using 

empathy to gain trust and then shutting it down to do harm. 


Suggested Reading 


Baldwin, “Infants’ Contribution to the Achievement of Joint 
Reference.” 


Jones and Klin, “Attention to Eyes Is Present but in Decline in 
2-6-Month-Old Infants Later Diagnosed with Autism.” 


Question to Consider 


Brain-to-brain communication has always been a popular science 
fiction trope, but with the development of new technologies, 
entrepreneurs are working hard to make this a reality. Considering 
what you know about pragmatic communication, what 

might be gained and lost by such direct communication that 
bypasses language? 
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12. 

MASTERING 
THE STRUCTURE 
OF LANGUAGE 


This lecture discusses how the mind 

is wired from birth to see structure 

in language. It is also wired to use this 
structure to organize and communicate 
information. As with other aspects 

of language development, this innate 
mechanism interacts with linguistic 
experience over childhood to allow the 
design of language to emerge. 
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Pattern Recognition and Hypothesis Testing 


The syntax and grammar at the core of a language rely on the ability 
to produce, recognize, and predict patterns. Pattern processing is 
one of the most widespread and useful mechanisms that humans 
innately possess. 


Infants constantly collect data and test hypotheses about their world. 
The general process underlying this cycle is a type of probabilistic 
learning called Bayesian learning. The basic idea is that we make 
predictions about the world based on a combination of two things: 
current evidence and prior knowledge. The power of Bayesian learning 
is that once we test a prediction or change our prior knowledge, it 
updates the probabilities of future predictions. This cycle is on a never- 
ending loop. 


Bayesian learning allows for a very fluid and dynamic process of 
educated guessing. Predictions can constantly be updated based on new 
evidence. Additionally, learning an important new piece of information 
can change a person’s prior knowledge, and this can radically change 
how he or she interprets patterns of evidence. Bayesian learning also 
allows for the creation of powerful, flexible rules about the world. 


Less Is More 


Bayesian learning is a nice example of a constraint on language 
acquisition that brings something extra to the learning process. 
However, some language-learning aids involve subtracting rather than 
adding. One example illustrating this is a hypothesis by Elissa Newport, 
aptly named the less-is-more hypothesis. 


Newport’s idea arose in response to a longstanding observation in 
language learning: Children learn language better than adults—a lot 
better. We know this from decades of research on second language 
learning (or L2 learning, for short), where children consistently 
outperform adults in terms of L2 pronunciation, morphology, and 
complex syntax. If you recall, Pat Kuhl had one explanation for this 
paradoxical finding: Children’s brains are much more plastic than adults 
brains, and this plasticity helps them keep their options open. 


? 
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Newport provides a different explanation: In addition to having brains 
that are more plastic, children possess a much more limited cognitive 
capacity than adults. Because adults are capable of encoding and 
retaining much more information than children, they learn linguistic 
structure in a more holistic way. In contrast, because young children 
have a much more limited memory system, they learn linguistic 
structure in a more compositional way. 


For instance, an adult may learn a whole word made up of 

three morphemes without realizing that it is composed of those 

three morphemes. For every new word that adults learn, they map 

the whole word onto an independent meaning. It is only after some time 
that they realize that individual morphemes are contained in the word, 
which causes them to try to analyze the parts after they have encoded 
the whole. This is a convoluted and cumbersome process familiar to 
anyone who has ever tried to learn a foreign language as an adult. 


In contrast, because young children can’t encode and remember as 
much as adults, they initially only remember parts of words. 


Over time, this sort of piecemeal learning will actually be more useful 
for the child to appreciate the morphological complexity of the 
language. In this way, processing less information allows children to 
learn more. 


The Role of Limitations 


Given that maturational limitations on memory are very useful 
in helping young children learn language, it’s worth asking what 
role these constraints played in the large-scale evolution of language. 


The constraints did not evolve specifically to help children learn 

the structure of language. We know this for two reasons. First, these 
cognitive limitations are very domain-general; they affect every aspect 
of a child’s mind, including perception, action, attention, emotion, 

and language. 


Second, these maturational limitations didn’t evolve for any purpose 
at all. These constraints are just a natural byproduct of human 
brains taking a very long time to develop. That is the unavoidable 
biology of altricial species. We have an extended period of cognitive 
and physical immaturity. There is a term in biology for phenotypic 
characteristics that are byproducts of some other aspect of an 
organism. They are termed spandrels, which was coined by the 
Harvard paleontologist Stephen Jay Gould. 


The alternative is to flip matters around, as Morten Christiansen 
and Nick Chater suggest doing. Instead of asking if maturational 
constraints on learning evolved in response to the structure of 
language, we might ask if the structure of language evolved in 
response to maturational constraints on learning. 


Christiansen and Chater’s argument is that because the basic 
biology of the human brain changes much more slowly than cultural 
artifacts like language, the brain is a more likely anchor for evolution 
than language. After all, humans were an altricial species with an 
extended period of immaturity for millions of years before language 
even entered the scene. This suggests that when language did arrive, 
it had a stable neural environment to adjust to. 
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A constant feature of that environment is that brains are in a form 
of extended immaturity while they learn language. Therefore, 
whatever structure language initially took in its evolution should 
have exquisitely adapted to that particular cognitive environment. 
Language evolved to be learnable in the context of the type of 
brain that was learning it. 


Language out of Thin Air 


It is no longer controversial to claim that there are innate 
mechanisms for language development. However, there is still 
debate over the nature of these innate mechanisms: Are they 
specific to language or more domain-general? Most scientists now 
believe that domain-general theories are the more parsimonious 
and persuasive explanation. However, this issue is far from resolved, 
and there are some empirical findings that remain 

active battlegrounds. 


One relevant area involves sign language. 
Before the 1970s, the medical community 
in the United States was not convinced 
that sign language was a real language. 
One implication of this view was 

that the medical community actively 
discouraged the teaching of sign 
language to deaf children. In its place, 
they advocated for lip reading. 


Unfortunately, this often had damaging 
consequences for both children and parents because 

lip reading is extremely hard and not very reliable. Despite the 
practice of not teaching sign to deaf children, many of these children 
created a language of their own. It was called home sign, and it was 
systematically documented by Susan Goldin-Meadow as part of her 
doctoral dissertation. 
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Home sign looks a lot like a conventional sign language. It has structure 
and consistency. Additionally, it bears uncanny resemblance among 
children who develop it. Individual signs differ across children, but 

the structure is very similar. This is true not only in the United States. 
Goldin-Meadow also observed a similar structuring in deaf children 
from Taiwan. 


One of the most striking shared structures of home sign is its syntax. 
Home signers in America and Taiwan both use an ergative syntax. 
Ergativity concerns how subjects and verbs are ordered ina 
sentence. Spoken English or Mandarin are not ergative languages. 
This is remarkable, and it raises the question of where the common 
structure derived from. 


One possibility was that it came from the co-speech gestures of the 
parents. Goldin-Meadow considered that, and she found that the 
gestures of American and Taiwanese parents are very different from 
each other. It was also likely that the similarity was not coming from 
some shared aspect of the environment, because America and Taiwan 
are very different culturally. 


The question remains: What gives rise to this commonality? 

The domain-specific theorists argue that there is some special 
module built into the human brain that allows language to unfold in 
a structured and consistent way, despite not receiving conventional 
input. This is compelling. The fact that something so language-like 
can emerge without any formal instruction makes a domain-specific 
mechanism sound attractive. 


The domain-general theorists see home sign as evidence that the brain 
is wired in a way that is conducive to learning language and also many 
other things. For example, there may be basic cognitive constraints that 
guide how meaning is mapped onto imagery and action. There may be 
strong social instincts to share and communicate perspectives with 
others. There may be probabilistic learning mechanisms that detect 
and impose regularity onto the world. These may combine to create 
language without being specifically designed for language. 
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In this case, it is difficult to know what the right answer is. Perhaps 
both views are correct. It is possible that language initially emerged 
from a domain-general mechanism in our evolutionary past, but once 
language arrived on the scene, it put selective pressure on brains to 
accommodate it. 


Suggested Reading 


Goldin-Meadow and Mylander, “Spontaneous Sign Systems Created 
by Deaf Children in Two Cultures.” 


Newport, “Maturational Constraints on Language Learning.” 


Questions to Consider 


Can you think of examples of how learning new vocabulary helps 
with learning syntax? What about examples of how learning syntax 
helps with learning new vocabulary? How can pragmatics help 
with both? 
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13. 
THE BRAIN AS A WINDOW 
INTO THE MIND 


A rare disorder called Capgras delusion causes people 
to believe that a close loved one has been replaced with 
a perfectly disguised imposter. This lecture discusses 
the neural mechanisms of Capgras delusion as a way 

to capture three basic principles of the human brain. 
Following that, the lecture looks at how advances in 
neuroscience methods are allowing scientists to ask new 
questions about how the brain works. 


The Biological Basis of Behavior 


The first principle this lecture discusses is neural specialization. 

During prenatal development, genes help to direct traffic, guiding 
which cells should migrate to which parts of the brain. Most of these 
cells are designed to go to very specific places and to serve highly 
specialized functions. Once they arrive, many require further input from 
the environment to activate their particular roles. Connecting to 

the Capgras delusion example: When a Capgras patient views her 
father as an imposter, avery specific part of the brain—the fusiform 
face area—responds to the face of her father. 


This principle of specialization works in tandem with the second principle: 
The brain is an elaborate network. Some basic functions of the brain are 
localized in just one area, like visual cells for processing color, edges, and 
motion. However, complex processes, like vision, must integrate these 
subprocesses, so an interconnected neural network is required. In a person 
with Capgras delusion, the FFA is undamaged, but the connections are 
broken. The exact mechanism involved in Capgras delusion is still being 
worked out, but it involves links between the FFA and other key brain areas. 


The third principle is that the brain is plastic and built for change. This is 

especially true early in life, but the brain is capable of change throughout 
a person’s lifetime. It had better be, or else you'd be in big trouble! Every 
time you learn something new—no matter how old you are—your brain 

changes. At the biological level, neural change is what learning is. 


This plasticity is integral to the first two principles. Even though genetics 
sets the basic blueprint for the brain, experience with the outside world 

is necessary to activate and fill in the details for many functions. 

For example, this is true in the FFA, where experience with particular faces 
is necessary for specializing cells to process those faces. 


Plasticity is also the key to recovery from broken connections in the 
brain. There are no drugs to fix brain damage. The main treatment is to 
undergo intense mental rehab. If a patient suffers from Capgras delusion, 
one treatment may be to repeatedly associate pictures of a loved one 
with positive emotions. Over time, this may allow weakened connections 
to get stronger, and in highly plastic young brains, it could even help 
create new connections to replace the old ones. 


13. THE BRAIN AS A WINDOW INTO THE MIND 


86 


Spatial Neuroimaging Techniques 


Spatial neuroimaging techniques answer the question of where 

brain activity occurs. The modern tools are called structural imaging 
techniques, but for a long time, researchers had to mostly rely on 
postmortem analyses of brain damage to map how different parts of 
the brain relate to different aspects of behavior. 


Structural imaging techniques take static snapshots of brain structures, 
and many of these snapshots can be compared to determine what areas 
are reliably associated with behavioral impairments. These techniques 
allow scientists to compile entire inventories of brain images to more 
precisely describe brain-behavior relationships. 


Another strength of these techniques is that they can look at brain 
structures immediately after damage, before there is any time for the 
brain’s plasticity to compensate for that damage. After all, traditional 
brain damage studies had to wait until the patient died to look at brain 
structure, which could happen decades following the actual damage. 
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However, there is one giant 
limitation of these structural imaging 
techniques. Because they are static 
snapshots of the structure of brains 
at rest, they cannot directly measure 
brain activity while it is actively 
engaged in some sort of behavior. 


This is where fMRI, or functional 
magnetic resonance imaging, enters 
the picture. In the early 1990s, Dr. 
Seiji Ogawa and his team added an 
important component to traditional 
MRI techniques. Standard MRIs 
measure magnetic properties of 
the brain to make a map of brain 
structure, but fMRIs are also able to 
measure real-time changes in brain 
function by recording the magnetic 
properties of blood flow within 
those structures. 


This technique led to an 
exponential increase in research 
on brain function and has given 
rise to several new fields, like 


The Power of 
the Brain 


In 2013, one of 
the three most 
powerful computers 
in the world—a 
supercomputer in 
Japan that could 


handle 10 quadrillion 
operations per 
second—required 
a full 40 minutes 
to simulate an 
approximation of 
what the brain does 
in just one second. 


cognitive neuroscience, clinical neuroscience, and neuroeconomics. 
These fields have made it possible to answer questions that were 
previously out of reach using data from structural imaging. For 
example, disorders such as Capgras delusion would likely be missed 
with structural imaging techniques alone. The structures may appear 
intact, but the breakdown is in how the structures function and connect 
with one another. Blood flow can show which parts of the network are 
working properly and which parts are compromised. 
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Building on the fMRI technology, researchers can use software to 
map connective tissue among brain areas to map out the connections 
in the brain. Diffusion tensor imaging (or DTI) is one such technique 
that can create maps of white matter tracts to better understand the 
information superhighways of the brain. 


Temporal Tools 


As useful as it is to understand where things happen in the brain, it is 
also important to understand when they happen. In terms of temporal 
resolution, techniques such as fMRI are slow. Current fMRI sensors can 
only see brain changes that happen on the order of seconds, which is 
a snail’s pace in terms of the speed of neural processes. 


To measure neural processing at this level of temporal resolution, 
neuroscientists must rely on techniques that are built for speed. 

One of the most well-established techniques is called event related 
potentials, or ERPs. ERPs are portions of the brain’s EEG signal, which is 
measured by an electroencephalogram. 


This EEG signal is produced by postsynaptic discharges of large groups 
of neurons in the brain that pass through the skull and are measured 

by electrodes on the scalp. An ERP is a time-locked portion of this EEG 
signal. Time locking means that an experimenter records only the brain’s 
repeated response to particular types of stimuli, like faces or words. In 
this way, the resulting electrical average reflects the neural activity that 
is unique to different stimuli. 


For example, a researcher studying a Capgras patient may be interested 
in the timing of how the patient processes the face of the supposed 
imposter. There is a typical time course and topography of a brainwave 
that is produced by a face, so if the ERP in response to the imposter’s 
face is different from the face of a non-imposter, it is possible to make 
inferences about what different type of neural process is involved when 
seeing the imposter. 
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Specifically, if the two brainwaves differ within 200 milliseconds of 
seeing the face, it would suggest an early event that involves low- 

level perceptual processing, such as how the brain initially categorizes 
the face. If the difference occurs much later, after roughly 500 
milliseconds, it would suggest a late event that involves higher-level 
cognitive processing, such as how the brain attaches emotional meaning 
to the face. 


The Best of Both Worlds 


In the past few decades, there have been technological innovations 

that allow neuroscientists to measure the timing and location of neural 
events. One of the most powerful—and expensive—techniques is called 
magnetoencephalography (or MEG). MEG uses a powerful magnet that 
is supercooled down to -452 degrees Fahrenheit. These cooled magnets 
are able to detect very small magnetic fields that are produced by 
electrical generators in the brain called dipoles. 


By measuring dipole activity, MEGs can measure the location of the 
electrical activity in small clumps of thousands of neurons as well as 
the millisecond-to-millisecond time course of that neural activity. Using 
this combined spatial and temporal resolution, an MEG could determine 
if a Capgras patient first processed two faces similarly in visual areas 
toward the back of the brain but then later differentiated them in 
frontal regions of the brain. This sort of pattern would lend support 

to the hypothesis that the patient’s problems in Capgras are more 
evaluative than perceptual in nature. 


One of the most exciting developments in neuroscience techniques 

is called neurostimulation. Unlike neuroimaging methods—which 

are designed to record brain activity—neurostimulation does 

what it sounds like it does: It stimulates brain activity. The idea is 

that neuroimaging can show researchers when and where brain 

areas are involved in some sort of process or behavior, and then 
neurostimulation can actively test whether that area is an actual causal 
mechanism of that process or behavior. 
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For example, transcranial magnetic stimulation (or TMS) uses 

handheld electrified coils to shoot pulses of magnetic fields through 

the skull into the surface of the brain. These magnetic pulses can 

either depolarize or hyperpolarize large patches of neurons in particular 
brain regions. Generally speaking, if the region is depolarized, it disrupts 
that brain region’s ability to do its job, but if it’s hyperpolarized, it can 
enhance that region’s ability to carry out its function. The disruptions 
are sometimes referred to as virtual brain lesions, and the 
enhancements are occasionally referred to as neural augmentation. 


If a researcher wanted to determine if Capgras delusion is caused by 
breakdowns in the right frontal lobe during processing, the researcher 
could test it in people who don’t have the disorder. The strategy 
would be to use TMS to disrupt (or create a virtual lesion) in this brain 
region—at just the right time—milliseconds after showing pictures of 
familiar faces to people who don’t have the delusion. 


If the right frontal lobe is a key mechanism for Capgras, this stimulation 
may cause problems in people’s ability to cognitively evaluate the faces. 
This would lend support for the theory that Capgras delusion is not 

a perceptual problem as much as it is a problem of attaching meaning 
to faces. 


Neurostimulation has gone to the next level with a radical technique: 
neurostimulation implants. Deep brain stimulation (DBS) was 
pioneered in the mid-1980s as a powerful new treatment for 
Parkinson’s disease. It involves implanting electrodes in the subthalamic 
nucleus, which is a structure deep in the brain that produces the 
neurotransmitter dopamine. The idea is if this part of the brain is 
stimulated, there will be more dopamine produced in the patient, and 
this increased dopamine can help to reduce some major symptoms of 
Parkinson’s disease. 
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Since this initial work with Parkinson’s, there have been attempts to 
place neural implants in other brain regions to treat problems like 
obsessive-compulsive disorder, chronic pain, and even depression. 
Some researchers are now using implants to record from a variety 
of brain sites to determine how and where different brain regions 
contribute to different brain functions. 


Suggested Reading 


Ramachandran, A Brief Tour of Human Consciousness. 


Question to Consider 


Most genes must be activated by environmental triggers from the 
immediate cellular environment all the way out to the external 
social environment. As a general rule, the more complex a behavior 
or trait is, the more it relies on the environment. What does this 
mean for claims about what is innate in the human mind? 
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14. 
HOW THE BRAIN 
COMPREHENDS LANGUAGE 


One day in 1939, an Englishman called Henry G. was 
admitted to the Bristol General Hospital in a state of 
unconsciousness after falling from a bus. Once Henry 
G. awoke, he gave the impression of having gone 
completely deaf from hitting his head. However, after 
closer investigation, it became clear that Henry G. had 
a much more peculiar deficit: He had lost the ability to 
understand human speech. 


Henry G. had a rare condition called auditory verbal 
agnosia, or pure word deafness. This lecture discusses 
how disorders like this one shed light on the neural 
mechanisms for language comprehension. It highlights 
the big three neural principles: specialization of function, 
the network property of the brain, and the power 

of plasticity. 


Dual Streams: Vision 


To better understand the journey that speech takes to create 
linguistic meaning in our heads, vision is a helpful gateway. Vision is 
one of the most studied processes of the brain, and neuroscientists 
believe that some fundamental aspects of vision are shared by many 
other systems, including language. 


When photons reach the eye, they are chemically transduced into 
electrical signals that are sent through subcortical relay stations to 
the back of the brain in the occipital lobe. The primary visual cortex, 
or V1, is located in the occipital lobe. That is the start of two journeys. 


From V1, visual information is split in two parallel streams, one going 
downward and one going upward. The one going down is called the 
ventral stream, and its function is to process the visual details of 
whatever thing a person is viewing. At the same time this is happening, 
the dorsal stream is processing a complementary piece of visual 
information: the location of the object. This pathway runs upward from 
Vi through brain areas specialized for processing motion and finishes up 
in the parietal lobe, where neurons analyze the relative location of the 
object to one’s own body. 


Parietal lobe 


Dorsal visual stream; 
this portion determines 
“Where is it?” 


Occipital lobe 


Ventral visual stream; 
this portion determines 
“What is it?” 
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These systems work in parallel to process visual information. Damage to 
one part of the system impairs one aspect of vision but not the other. 
Damage to the front end of the ventral stream disrupts the ability to 
correctly identify what an object is, but it spares the ability for the 
dorsal system to determine where it is. The opposite is also true. 
Neuroscientists believe that the primary function of having these two 
parallel visual streams is that vision can specialize its processing while 
also ensuring that it is very fast. 


Dual Streams: Language 


This neural strategy of dividing and conquering is so effective 

that it is now believed that it is also used in language processing. 
Although originally hypothesized by Carl Wernicke himself in the late 
1800s, this dual-stream hypothesis for language has gained new life 
a century later with the advent of functional neuroimaging methods. 
Most notably, a version of this model was reinvigorated in 2007 by 
the cognitive neuroscientists Gregory Hickok and David Poeppel. 


When speech sound waves travel toward us, they are funneled from the 
external ear, called the pinna, inward to the eardrum. The pinna 

funnels and amplifies speech sound waves before they hit the 

eardrum. The vibrations there are mechanically transduced into action 
potentials in the cochlea and are then sent via subcortical relay stations 
to A1 in the temporal lobe. A1 is topographically organized for sound 
input, just like V1 is for visual input. 


From Ai, all processes move to the surrounding superior temporal 
cortex, which has been traditionally called Wernicke’s area. It has 
multiple functions. One of the earliest stages of processing after 
A1 is to recognize acoustic features of words. It was once assumed 
that this processing occurred only in the left hemisphere, but it is 
bilateral. For example, neuroimaging scans show that patients with 
lesions to only the left superior temporal lobe can still recognize 
words as words. 
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However, bilateral damage to this part of the brain is devastating for 
word recognition. In fact, this is most likely what caused Henry G.’s pure 
word deafness. He could not hear words as words. For him, it was all 
gibberish, and modern neuroimaging scans now show the culprit to be 
bilateral damage to this brain region. 


Researchers have used fMRI on non-brain-damaged participants to 
show that this region may be specialized for the sounds of language. 
Many studies have now found that the bilateral superior temporal lobe 
is more active for speech stimuli than complex non-speech sounds, 
suggesting that it is indeed one of the first processing points that treats 
speech as a special type of acoustic signal. 


This is also the first stage of processing that distinguishes phonological 
features within spoken words themselves. Hickok and Poeppel argue 
that the left hemisphere analyzes the 
fine-grained details of phonological 
forms, whereas the right hemisphere 


does amore holistic analysis. : 
y 100,000 Miles 


The Ventral Stream Here is an interesting 
fact, via the 
psycholinguist Julie 
Sedivy: It is estimated 


From the superior temporal lobe, 
the two streams split. One travels 
ventrally and the other dorsally. 


These two pathways ride along two that the average 
well-established white matter tracts. 20-year-old has 
Regarding the ventral speech stream, approximately 100,000 
the transitional area connecting miles of these white 


the posterior superior temporal matter tracts. 
lobe and the middle temporal gyrus 
is where words are recognized as 
having meaning. Damage to this area 
produces receptive aphasia, which 
was traditionally called Wernicke’s aphasia. Unlike patients with pure 
word deafness, these patients can produce and hear speech as speech, 
but the speech lacks appropriate meaning. 
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With the advent of spatial neuroimaging methods, we now have 
hundreds of studies implicating this region as a mechanism for 
attaching meaning to words. Temporal neuroimaging methods 

have shown that this semantic processing occurs milliseconds after 
phonological processing at earlier stages. As a general rule, as 
information travels farther toward the anterior part of the temporal 
lobe, the neural mechanisms become increasingly specialized for 
processing more complex semantic information. 


Another notable feature of moving toward the anterior portion 

of the ventral stream is that it becomes increasingly more left- 
lateralized. As it becomes more specialized for complex semantics, 
the left hemisphere takes on more and more of the processing 
burden. The right hemisphere is still involved in processing meaning, 
but it does a much more coarse-grained analysis. 


Activation Zones 


In one of the largest meta-analyses on the brain’s semantic system 
for language, Jeffrey Binder and his team reviewed more than 500 
PET and fMRI studies and found that there were a few main areas 

of activation. One area was located in the cortical areas surrounding 
the left auditory cortex, including the left temporal lobe and parts 
of the inferior parietal region. Connections to these areas are 
thought to activate visual imagery. For instance, one mechanism for 
remembering the meaning of the word key is to reactivate the visual 
perception of images of keys. 


However, it’s important to note that semantic memory and visual 
perception do not overlap fully. There are cases of brain damage 
to these visual areas where object recognition is completely lost, 
but patients can still correctly understand the meaning of words 
associated with those objects. 
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Another important zone is located in the motor and premotor cortex 
in the frontal lobe. As shown in the work of Friedemann Pulvermiiller, 
when someone hears the word kick, the foot region of the primary 
motor cortex becomes active, but when they hear throw, the hand 
region gets involved. Again, the overlap is not complete: There are 
cases of brain damage where the motor system is compromised, but 
linguistic knowledge about what objects do is preserved. 


In addition to neocortex activation, limbic regions also get in on 

the act of attaching meaning to words. The part of the subcortex 
responsible for encoding and retrieving memories, the hippocampus, 
is constantly active when searching for word meanings. And 

because many words have episodic and emotional associations, 
parts of the cingulate cortex become active at attaching those 
associated meanings. 


In 2016, Jack Gallant and his team conducted a methodologically 
innovative study measuring the brain’s semantic processing not of 
isolated words but of whole narratives. Gallant’s team found that 
words activated practically every square inch of the brain’s surface. 
The activation maps were highly organized. 


Dorsal stream 


Broca’s area 


Wernicke’s area 


Ventral stream 
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After the meaning of words is assembled from all corners of the brain, 
the information is channeled toward the final stop on the ventral 
pathway: the left inferior frontal gyrus (or IFG). The left IFG is involved 
in the integration of multiple pieces of information. In this case, the 
information that must be integrated is the meaning of words and 
their syntactic context. This final step is necessary for comprehending 
language in its full creative complexity. 


Studies have produced direct evidence that the left IFG is constantly 
monitoring and updating the syntactic context of language, which is not 
only useful for correctly understanding complex meanings but also for 
predicting meaning that is coming down the stream. 


The Dorsal Stream 


While information moves up and down the dorsal stream, a second 

set of processes is ongoing in the dorsal stream. It is helpful to think 

of words as objects: The function of the dorsal language stream is to 
act on those objects. This is not as much of a stretch as it seems. For 
example, one way for the visual system to know an object is to interact 
with it. In a similar way, one way for the auditory system to know a word 
is to produce it. For the dorsal stream, knowing is doing. 


The significance of this mechanism is clearest when considering how 
infants learn to produce language in the first place. When a baby 
babbles, it is coordinating its motor system to match what its auditory 
system is hearing. Humans are born with an ability to hear all phonemes, 
but the motor system is much less advanced. It requires months of 
practice to learn how to produce the phonemes it hears. This extensive 
experience lays the tracks for the dorsal route in the first few 

years of life. 


The first stage of the dorsal route goes from Wernicke’s area to the 
left Sylvian parietal temporal region (or SPT). SPT gets its name from 
straddling the Sylvian fissure, which divides the temporal and parietal 
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lobes. As a general rule, the dorsal stream is much more left-lateralized 
than the ventral stream, and this is because motor commands for 
producing speech are located mostly in the left hemisphere. 


The SPT takes the phonological signal from earlier stages and 
rehearses it. Another function of SPT is to learn new vocabulary items. 
Interestingly, rehearsing happens mostly for more complex words. 

For words composed of relatively simple or highly familiar phonemes, 
this stage seems to be skipped, and it is handled downstream toward 
the endpoint of the dorsal route. 


The main path for this downstream connection is a tract of white 
matter fibers called the arcuate fasciculus. The final stop of these 
fibers is the left prefrontal cortex and the IFG. These regions are 
heavily involved in overt speech production. In fact, damage to the 
left IFG is what produces Broca’s aphasia. 


Broca’s area—and the left IFG more generally—also serves a syntactic 
function. It has long been observed that Broca’s aphasics have deficits 
in comprehending complex sentences. This syntactic function of the 


left IFG has been corroborated in many functional imaging experiments. 


Additionally, developmental studies have shown that children’s syntactic 
abilities are positively correlated with the strength of connection 
between the left IFG and more posterior parts of the ventral stream. 


These developmental data suggest that the IFG is strongly shaped 
by practice and experience, much like the FFA with face processing. 
In fact, just like the FFA, the left IFG may be more domain-general 
than originally thought. There is growing evidence that it processes 
syntactic patterns in language and is sensitive to complex patterns 
outside of language, such as music. 


A domain-general view might explain why the dorsal and ventral routes 
converge in the left IFG. Peter Hagoort of the Max Planck Institute for 
Psycholinguistics argues that the left IFG is a “unification site” that 
combines information across semantics, syntax, and pragmatics too. 
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In other words, its function is to gather all the relevant pieces of 
information in a communicative utterance and then piece it together 
into a coherent package. 


Just like the ventral stream, the unified meanings generated by the 

left IFG feed backward along the dorsal stream. This helps earlier 
dorsal stages to quickly connect motor representations to the acoustic 
signal coming from A1. 


Suggested Reading 


Hickok and Poeppel, “The Cortical Organization of Speech 
Processing.” 


Huth, de Heer, Griffiths, Theunissen, and Gallant, “Natural Speech 
Reveals the Semantic Maps That Tile Human Cerebral Cortex.” 


Questions to Consider 


Greg Hickok and David Poeppel’s dual stream model explains 
how the brain balances speed and accuracy in processing 
language. Using the 3-D framework, how might this moment-to- 
moment mechanism relate to language change over the historical 
timeframe? For example, how might these two complementary 
pressures—speed and accuracy—gradually change phonetics, 
morphology, or syntax as language is passed down over multiple 
generations of users? 
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15. 
HOW THE BRAIN 
PRODUCES LANGUAGE 


This lecture explores the mechanisms involved in 
language production. The process of speaking involves 
a complex network of specialized brain regions honed 
by genetics and experience. 


What’s a Thought? 


The spark that prompts someone to speak in the first place is a bit of 
a mystery because it is internally generated: It’s the thing in your head 
you want to communicate. 


Most of us would call these things thoughts. However, modern-day 
cognitive scientists are increasingly skeptical about the conventional 
notion of thoughts. One problem is that no one has ever seen one 
directly. A thought is just a convenient construct that we use to explain 
things we subjectively experience. 


Another problem is that a thought is typically viewed as a stable 

and discrete thing. However, from a neural point of view, thoughts 
are dynamically shifting and multidimensional. A final reason that 
traditional notions of thought are on shaky ground is that we don’t 
know when a thought occurs. It is unclear if a thought occurs only 
when were consciously aware of it, or if it exists in some form before 
subjective awareness. 


Still, despite not knowing what thoughts are—or when or how 

they occur—many neuroscientists believe that the brain generates 
something in advance of our conscious awareness. For instance, 
readiness potential is an electrical brain response first observed in 

an EEG experiment done in 1983 by Benjamin Libet. The experiment 
measured electrical activity on the scalp as people voluntarily reached 
for objects. 


The main finding was that a distinct brainwave—the readiness 
potential—preceded awareness of voluntary reaching by several 
hundred milliseconds. Itzhak Fried and his colleagues have since 
replicated this interesting finding in epilepsy patients with electrodes 
implanted in their frontal lobes. 


There is a range of interpretations of these findings, but the results 
clearly show that some sort of unconscious neural activity precedes 
action. If this is the case for relatively simple things like reaching, it 
suggests that for more complex actions—like using language—there 
may be a large and clandestine neural network operating behind 

the scenes. 
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Two Stages of Speech Production 


Spoonerisms, a type of speech error, are named after Reverend 
William Archibald Spooner, don of Oxford College around the turn 

of the 20th century. Spooner was famous for his linguistic slip-ups. 
For instead, instead of using the phrase “a loving shepherd,” Spooner 
is said to have given us: “The lord is a shoving leopard.” 


Mistakes like this reveal that speaking unfolds according to a 
premediated plan: What else could explain why parts of words that 
should appear later in an utterance mistakenly show up earlier? 
Spoonerisms are cousins to a different preplanning error involving 
swaps of entire words within an utterance. For example, imagine 
someone saying: “Can you get some kids for the snacks?” 


Psycholinguists argue that these two types of speech errors represent 
two different stages of speech planning. Exchanging entire words 
within syntactic categories provides a glimpse into the initial abstract 
stage of conceptual planning. The technical word for these preplanned 
abstract concepts is emma, which in ancient Greek means “premise.” 
Once the lemma has been planned, the next stage begins. The second 
step involves putting those abstract ideas into the proper phonological 
forms, and Spoonerisms happen during this step. 


But still, even if run-of-the-mill priming explains these findings, they 
clearly show that linguistic planning stages involve more than just the 
neutral nuts and bolts of language. This opens the door to a wide range 
of information being part of the production process. 


Neural Models of Speech Production 


For more clarity on the different stages of language production, 
it is helpful to shift to the neural level of analysis, beginning with 
what brain damage can tell us. Wernicke’s aphasia is traditionally 
viewed as a problem with language comprehension, but that has 
proven to be overly simplistic. 
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One of the telltale signs of Wernicke’s aphasia is called a paraphasia. 
There are many forms of this. There are phonemic paraphasias, which 
involve missaying parts of words. Semantic paraphasias, meanwhile, 
involve perfectly articulated words showing up in semantically 
incorrect places. 


A patient with Wernicke’s aphasia can have both types of paraphasia, 
but they often have only one. In these cases, it has been hypothesized 
that phonemic paraphasias are caused by damage in the dorsal stream 
involved in phonological processes, and semantic paraphasias are 
caused by damage to the ventral stream involved in meaning. 


It’s difficult to reliably map these areas with functional neuroimaging 
because of all the plasticity that takes place after brain damage. 
However, there are some more direct techniques that A 
have nicely shown this functional specialization. 


Earlier, Dr. Wilder Penfield developed 

a procedure to map the brains of epilepsy 
patients before extracting tumors. He opened 

up the skull and electrically stimulated certain 
parts of the cortex to determine localized 
functions. That technique continues to be 

the most direct way that neuroscientists can 
map the specific functions of the human brain. 

In 2010, David Corina and his team used the same 
technique to map language functions of more than 100 

epilepsy patients having tumors removed from their left hemisphere. 


While lying fully conscious on the operating table—with their brains 
exposed— patients were asked to name objects presented in pictures. 
While naming the objects, the surgeon electrically stimulated the left 
hemisphere to determine a safe place to operate. 


There were many interesting findings, but the most relevant is that on 
average, stimulation to the dorsal language stream produced more 
production errors such as phonemic paraphasias. Stimulation to the 
ventral stream produced more semantic paraphasias. This finding 

fits very well with the two-stage model of semantic and phonological 
speech planning. 
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Broca’s Area 


The brain’s left hemisphere is home to the most well-known mechanism 
for speech production: Broca’s area, which is where speech sounds are 
turned into speech actions. Fitting with this course’s 3-D model, the 
ability to speak comes from a combination of genetics and experience. 


Genes dictate that the posterior part of the frontal lobe will ultimately 
specialize for planning and producing speech movements. Auditory 
experience with one’s native language over development allows those 
cells to eventually mimic what they hear. 


In patients with Broca’s aphasia, the ability 
to link speech actions to speech sounds is 
compromised, and it creates a telltale kind 
of dysfluency. If Wernicke’s aphasia is a 

breakdown in knowing the right thing to say, Congenital 
Broca’s aphasia is a malfunction in physically 


Congenital 
Deafness 


ane deafness 
— makes it so 
Broca’s aphasics are painfully aware of their hard to speak 


condition, whereas Wernicke’s aphasics seem 
much more oblivious. When Wernicke’s 
aphasics use a paraphasia, they act as if it 
makes total sense. Broca’s aphasics are much target for 
more sensitive to their listeners. Often, they Broca’s area 
will resort to writing things down for people to hit. 

to make themselves understood. 


because there 
is no auditory 


Retraining the Brain 


The most impressive demonstration of neural plasticity in the human 
brain comes from an operation called complete hemispherectomy. This 
is a radical, last-resort operation to treat patients (mostly children) who 
have life-threatening seizures caused by abnormalities on one side of 
the brain. The procedure involves opening up the skull and surgically 
removing the entire affected hemisphere from the patient’s head. 
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These procedures were pioneered in the early to 
mid-1900s, but they were revamped and refined 
in the 1980s by surgeons at Johns Hopkins 
Hospital. (One of the people responsible was 
America’s most famous brain surgeon: the 
2016 presidential candidate Dr. Ben Carson.) 
Today, doctors can isolate smaller regions 
within a hemisphere, but in some cases, a full 
extraction is still necessary. 


When the entire left hemisphere is removed, the 
immediate consequences are dramatic. Assuming 

the patient had left-lateralized language (which most people 

do), the removal of that hemisphere completely shuts down speech—at 
least temporarily. There is much individual variation, but many patients 
can start recovering language functions within weeks. 


This sort of recovery is possible because the right hemisphere 
undergoes extensive plasticity and ultimately adopts the functions 
of Broca’s area. That process is amazing, and a large factor in making 
it happen is the fact that the right hemisphere already does a large 
amount of work during language production. 


While Broca’s area is working on articulating words, there is a 
corresponding region in the right hemisphere working on a higher level. 
This homologous brain region simultaneously provides a rhythmic 
structure that connects spoken words across an utterance. Linguists call 
this acoustic link prosody. 


The takeaway point is that when the right hemisphere takes over the 
functions of Broca’s area, it’s definitely not a blank slate. Because the 
right and left hemispheres normally communicate constantly over a 
bundle of fibers called the corpus callosum, each hemisphere is well 
acquainted with what the other is doing. This means that even though 
the right hemisphere is designed for rhythmic aspects of speaking, 

in extreme cases, it can repurpose part of itself to be more like the 
left hemisphere. 
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This sort of plasticity is most prevalent in children, but the brain 
maintains some plasticity throughout life. This opens the door to 

a number of therapies for people who suffer brain damage as adults. 
The standard treatment for Broca’s aphasia is repetitive picture naming, 
which mostly targets the damaged left hemisphere. Speech therapists 
have also tried some more innovative approaches involving the 

right hemisphere. 


One novel approach capitalizes on hand gestures. Broca’s aphasics 
instinctively produce hand gesture as a form of compensation. 

By drawing heavily on the right hemisphere, the spatial and temporal 
properties of gesture can help aphasics find words and say them with 
a more natural prosody. 


A second creative therapy capitalizes on the right hemisphere’s penchant 
for melodies. Back in 1904, Dr. Charles Mills published an account in 

the Journal of the American Medical Association about an aphasic 
patient who learned to sing his words instead of speaking them. 

Others have made similar observations, and they have ultimately led to 
an official musical treatment called melodic intonation therapy, which 
was developed in the early 1970s. 


Suggested Reading 


Petersen, Fox, Posner, Mintun, and Raichle, “Positron Emission 
Tomographic Studies of the Processing of Singe Words.” 


Schlaug, Marchina, and Norton, “Evidence for Plasticity in White 
Matter Tracts of Chronic Aphasic Patients Undergoing Intense 
Intonation-Based Speech Therapy.” 


Questions to Consider 


The readiness potential reveals the time course of thoughts that 
precede actions. However, can’t producing actions also produce 
new thoughts that did not previously exist? Can you think of any 
examples of this? 
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16. 

DANCING BRAINS: 
THE SOCIAL SIDE 
OF LANGUAGE 


One of the biggest themes in this 
course is viewing language as a 
system. This lecture expands that 
system beyond individual brains. 
It looks at language from the 
vantage point of brains—highly 
social brains—interacting with 
one another. 


Changing the Question 


Traditionally, many psychologists and neuroscientists have studied social 
interactions by focusing mainly on the individual as the appropriate unit 
of analysis. The question has been: What is inside an individual’s head 
that allows for social communication with others? 


Toward the latter part of the 20th century, scientists interested in highly 
complex social activities, like language, began expanding this narrow 
focus. Instead of zooming in on the individual, many now focus on the 
social activity per se. 


Brains in the Mirror 


Neural mechanisms allow these joint activities. Simulation theory 

is the idea that the brain processes something by reactivating 

the parts involved in directly experiencing it. For example, there 

is not much difference in neural activity when a person bites 

an apple, imagines biting an apple, or imagines someone else biting 
an apple. This sort of shared simulation has been shown in many 
fMRI studies by Marc Jeannerod and Jean Decety, and it suggests 

a possible neural mechanism for how people subjectively understand 
the perspectives and actions of others. 


Simulation theory can explain what happens in individual heads and 
how two heads become linked as one. A well-known mechanism 

for making these neural links comes in the form of mirror neurons. 
Originally discovered in the prefrontal cortex of monkeys by Giacomo 
Rizzolatti, the mirror neuron system in humans is activated both when 
a person produces an action, like reaching for an object, and also 
when a person observes someone else produce that same action. 


Over the years, our understanding of the function of this mirror system 
in humans has greatly expanded. It is responsible for our understanding 
the intentions of actions and feeling the pain of others. It is also now 
being implicated in autism spectrum disorder, psychopathy, altruism, 
and even addiction. 
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One of the most misunderstood aspects is where they come from: Are 
they innate or a product of learning? Here, the 3-D framework can be 
useful, using the motor mirror system as an example. When very young 
infants view the reaching behaviors of others, the mirror system won’t 
be triggered. That is because infants must first have enough experience 
over development to master reaching for themselves. 


Only when reaching becomes part of their repertoire can babies 
activate the mirror properties of the neurons. Only a fraction of motor 
neurons have mirror properties, so it seems that just a subset of 

them have innate mirroring potential. In this way, the answer of where 
mirror neurons come from requires us to consider mechanisms on 
multiple levels and different timeframes: The system is the product 

of genes conserved over evolution, and plasticity and experience 
during development. 


Brains Resonating Together 


Since the turn of the millennium, the field of social neuroscience has 
progressed well beyond mirror neurons. Methodologies have advanced, 
allowing for more dynamic questions about how people synchronize 
with one another in real social interactions. For example, rather than 
just measuring one person’s brain in response to a social stimulus, it is 
now possible to simultaneously measure the brain activity of multiple 
people engaged in a joint activity. 


One of the first studies to pull this off was done in 2011 by Ulman 
Lindenberger and his team at the Max Planck Institute for Human 
Development in Berlin. They measured the EEG activity of pairs of 
guitarists playing short melodies together. The main finding was that the 
two guitarists synchronized their brains not only while they were playing 
together but also just before they began to play. The most synchronized 
frequency range was a range associated with voluntary control of motor 
actions. This means that the brains of the guitarists were on the same 
page even before they started playing. 
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This doesn’t show that neural synchrony is the cause of playing in sync 
with one another. Perhaps things like mutual eye gaze, head nods, 

and foot tapping to a beat are what synchronized their brains before 
playing and while they played. At the very least, though, this innovative 
study shows that brain synchrony is a valid neural marker of jointly 
coordinated social interactions. 


If neural synchrony is not the cause of social coordination, what is? 
One answer comes from an ambitious project that measured brain 
synchrony in a high school classroom over the course of a whole 
semester. David Poeppel and his team at the Max Planck Institute 
for Empirical Aesthetics in Munich had 12 students and their teacher 
all wear portable EEG headgear over the span of 11 sessions of a 
biology class. 


There were a number of interesting findings, but the most relevant 
concerns the alpha frequency of brainwaves, around 8 to 12 Hz. 
(Alpha readings reflect how deeply someone is paying attention.) 
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This simultaneous alpha frequency of all 12 students was most 
synchronized when they reported being more cognitively and socially 
engaged in class. This suggests that a common focus of attention may 
be what unites brains to become synchronized with one another. 


The Predicting Brain 


Again, neural synchrony is not the driver of joint engagement, but 
even if it is not the immediate mechanism, it may ultimately serve 
a downstream function of facilitating social coordination. Just like 
a bell continues to resonate long after it has been struck, neural 
synchronization may have enduring 
social vibrations. 


Traditional models of communication 
are static: There is a sender, and 
there is a receiver, and they serially 
transmit information in an orderly 
fashion. However, real face-to-face 
communication is nothing like this; it’s 
almost always much more dynamic 
and messier. As the Scotland-based 


Birds 
Communicating 


Birds are one of 
the only other 
species born not 


psychologists Martin Pickering and knowing how to 
Simon Garrod put it, communicators communicate with 
are “moving targets,” and coordinating their kind. Like 
between them is much more like a humans, they must 


dance than a transmission. ; 
learn it. 


In their model, Pickering and 

Garrod view this tangled overlap 
between speaking and listening not as 
a flaw but as an actual design feature 
that allows communication to work so 
well. Consider all the overlap in a real, everyday conversation: imitating 
head nods and hand gestures, mirroring facial expressions, and so on. 
These behaviors align people’s brains, and this resonates over time to 
help predict what will happen next in the interaction. 
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People don’t always make the right prediction, but that’s OK, because 
we can easily adjust on the fly. These spontaneous corrections 

are what linguists call conversational repairs, and people do 

them constantly. 


A Social World in Our Heads 


When we use language, we draw upon a large reservoir of what is 
mutually known among speakers. The Stanford linguist Herb Clark calls 
this mutual knowledge common ground. He argues that without it, 
successful communication is impossible. 


In the most extreme example, communicators must have common 
ground in what language they speak, but there are many other nuanced 
layers, including cultural factors, levels of expertise, knowledge of 
terminology, and so on. 


In one experiment, Thomas Holtgraves manipulated whether indirect 
requests were spoken by people who were higher in the social hierarchy, 
like a company boss, or more equal, like a coworker. The common 
ground in this case is that it is much more socially permissible for bosses 
to ask people to do things for them. 


That is what Holtgraves found. Participants processed the utterances as 
indirect requests more often and much faster when the speaker was of 
a higher status. Additionally, people seemed to bypass the literal content 
of the message more quickly in this higher-status condition too. This last 
part is particularly interesting because it suggests social information 
seemed to be built into the original message. 


Social Surrogates? 


Human laughter is highly social in nature. In the 1990s, Robert Provine 
did a series of studies on this subject. Provine found that humans are 30 
times more likely to laugh in a social context than when alone. People 
laugh not necessarily in response to something funny; laughter most 
often springs from just sharing certain experiences with other people. 
Lastly, Provine found that laughter is highly contagious. 
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One function of shared laughter is to socially bond with other people. 
There seems to be something special about sharing a good laugh that 
brings people together. However, any sort of social mirroring can 
function in a similar way. 


For instance, Franny Spengler and her colleagues had pairs of people 
play pantomime games where they either mimicked or did not mimic 
each other’s actions. Dyads that mirrored one another produced higher 
levels of a hormone called oxytocin, which is involved in social bonding. 
This suggests that there may be something biologically special about 
synchronizing behaviors with others. Being in sync may bring out our 
social best. 


Suggested Reading 


Dikker, Wan, Davidesco, Kaggen, Oostrik, McClintock, Rowland, et 
al., “Brain-to-Brain Synchrony Tracks Real-World Dynamic Group 
Interactions in the Classroom.” 


Pickering and Garrod, “An Integrated Theory of Language 
Production and Comprehension.” 


Questions to Consider 


Charles Darwin claimed that laughter is not unique to humans, and 
recent research has shown that dogs, chimps, and even rats emit 
distinct sounds in response to certain types of pleasure. Do you 
think that laughter plays the same social function for these animals, 
or is there something unique about the pragmatics of human 
laughter? What might someone like Michael Tomasello say about 
this? 
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HOW WRITING 
TRANSFORMED THE MIND 


The written word is a technology that has changed 
brains, elevated minds, and transformed civilizations. 
Even though our brains are not specifically designed for 
this technology at birth, prolonged and explicit exposure 
to it can repurpose parts of the brain intended for other 
things. In the end, these cobbled-together parts combine 
to create a neural network that eventually becomes 
dedicated to reading and writing in our mature brains. 


Reading Rewires the Brain 


There are two main parts of the story of reading and writing in the 
brain, and both of them take place in familiar brain regions. The first 
part of the story involves the homologue to the fusiform face area 
(FFA). The FFA is located in the fusiform gyrus, which spans the ventral 
part of the occipital and temporal lobes. It is a bilateral structure, which 
means it traverses both the left and right hemispheres. In the case of 
face processing, the FFA is right-lateralized, which means that the right 
hemisphere does most of the work. 


Its corresponding homologue is 

over in the left hemisphere. Stanislas 
Dehaene has dubbed that region the 
visual word form area. It is specialized 
for processing written words. This 
makes sense given what we know 
about the left hemisphere generally: 
It is specialized for sequential and 
componential processing. 


The Birthplaces 
of Writing 


Most scholars trace 
the invention of 


writing independently 
back to at least two 
different civilizations: 


Both hemispheres are still involved in 
processing faces and words, but the 
two sides approach them differently. 
The right hemisphere does more of Mesopotamia and 
the work when it comes to faces, and Egypt. 

the left hemisphere does more of the 
work when it comes to written words. 


The second part of the reading 

network involves the ventral and 

dorsal route for language processing. When children are learning to 
read English, for example, they spend a lot of time trying to sound out 
letters in words. We know from brain damage and neuroimaging studies 
that this overt sounding out during reading activates phonological 
mechanisms running up and down the dorsal stream. Even after children 
stop overtly reading aloud, they continue to covertly simulate this 
production in the same region. 
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Bradley Schlaggar and Bruce McCandliss explain that through massive 
repetition and neural plasticity, this sounding-out process gradually 
helps the ventral and dorsal streams of speech processing connect with 
the visual word form area. This neural network becomes established at 
different times for different people, but nobody comes into the world 
with such a network. It is not innate, and it requires experience with 
written objects to shape it. 


Dyslexia 


Even though this neural network for reading is not innate and must be 
built over time, that does not mean there are no genetic components. 
Dyslexia is one big reason why we know that reading is partly 

genetic. Dyslexia is common and is conservatively thought to affect at 
least 3 percent of the world’s population. It’s a very specific disorder 
that appears unrelated to overall intelligence. 


The National Institute of Neurological Disorders and Stroke defines 
dyslexia as “difficulty with phonological processing, spelling, and/ 
or rapid visual-verbal responding.” Note these two pieces: sounding 
out words and visually mapping letters to sounds. Those involve 
the reading network. 


The heritability of dyslexia is estimated to be around 60 percent. 
This means that in the entire population of readers, 60 percent 
of the variance in the behavioral manifestation of dyslexia can be 
explained by variation in genes across individuals. 


There are genes for motor control and visual processing abilities as 
well as genes for linking up those two systems. When confronted with 
the technology of the written word, the brain does its best to wire up 
the reading system, but in the case of dyslexia, there is some genetic 
malfunction with parts of that network. This does not mean there are 
genes for dyslexia. It does mean there are genes that get co-opted for 
the system. 


Synesthesia 


When looking at a written word printed all in black, a small percentage 
of people do not perceive that word as written in black. Instead, they 
see it as a rainbow of colors, with each letter having its own distinct 
hue. This phenomenon is known as grapheme-color synesthesia, and 
it is part of alarger phenomenon in which the brain has unusual cross 
talk during certain sensory experiences. This cross talk comes in many 
forms: Some people with synesthesia hear colors, others smell sounds, 
some feel numbers, and a few even taste words. 


Synesthesia is rare in the general population. Conservative estimates 
are that only about 1 in 2,000 people have some version of it. 
Interestingly, there seems to be a much higher incidence among 
artists and other creative people. 


Letters evoking colors is one of the most common types of synesthesia. 
Often, these color combinations are experienced emotionally or seen by 
the synesthete to be beautiful or ugly. 
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One well-established theory by V. S. Ramachandran and E. M. Hubbard 
is that there are crossed wires in the fusiform gyrus connecting the 
visual word form area to the nearby V4 area. V4 is in the secondary 
visual cortex and is specialized for color processing. There are clusters 
of cells dedicated to processing every color a person experiences. 

The theory posits that in infant brains, there are exuberant 
connections between this area and the neighboring visual word form 
area, and over typical development, plasticity prunes them away. 
However, this does not happen in synesthetes, and the cross-wired 
connections remain. 


Other theories downplay structural plasticity and focus more on 
functional plasticity. Even in neurotypical brains, there are connections 
between the visual word form area and V4 as well as many other areas. 
In fact, there are many silent connections between adjacent areas of 
the brain. These are called latent synapses. 


In neurotypical brains, high-level inhibitory mechanisms dampen 
communication along these latent synapses. Some scientists 

think things are different in synesthetes. The psychologists Peter 
Grossenbacher and Christopher Lovelace argue that a synesthete’s 
inhibitory mechanisms are weakened, and this is why there is cross 
talk between adjacent areas. Some compelling evidence to support 
this is that hallucinogenic drugs, such as LSD, decrease inhibitory 
mechanisms in the brain, which can induce massive synesthetic 
experience. The logic is that because these drug effects come on so 
fast, it would be impossible that new neural connections are created 
on the spot. 


One problem with both of these theories is that they can’t truly explain 
synesthesia involving links between nonadjacent brain areas. For 
example, some people experience certain words with particular smells, 
and because the brain areas responsible for both of those senses are so 
far apart, it is hard to see how they became linked up in the first place. 
There is some promising evidence that some of these links are genetics- 
based, but researchers have not yet been able to map specific genetic 
mechanisms onto specific types of synesthesia. 
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Regardless, the takeaway point is this: Both synesthesia and dyslexia 
nicely illustrate the network property of the brain. While dyslexia 
shows how the wiring between typically connected regions can be too 
weak, synesthesia shows how the wiring between atypically connected 
regions can be too strong. Just like typical reading development, both 
show how genes and experience combine with plasticity to ultimately 
create neural structures dedicated to the written word. 


Suggested Reading 
Borges, “The Library of Babel.” 


Dehaene and Cohen, “The Unique Role of the Visual Word Form 
Area in Reading.” 


Ward, “Synesthesia.” 


Questions to Consider 


Research by Linda Henkel has shown that compared to simply 
observing something, taking photographs of that thing makes 
people forget it more easily. Why might that be? Do you think the 
same thing happens when you put your thoughts into writing? Why 
or why not? 
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18. 
SIGN LANGUAGE: 
LANGUAGE IN OUR HANDS 


The World Federation of the Deaf estimates 

that worldwide, more than 70 million people 

use sign, which comes in many distinct forms, as 
their native language. This lecture explores that 
often-overlooked form of language. Exploring 

a version of language that operates in a different 
modality than speech can give a wider and deeper 
appreciation of what language is. 


Sign as Language 


The lecture begins by breaking down sign language according to our 
five linguistic components: pragmatics, syntax, semantics, morphology, 
and phonology. 


Starting with pragmatics, it is clear that the deep need to share 
information with other humans has always been a driving force in the 
use of sign. For example, there was a flourishing community of signers 
on the island of Martha’s Vineyard from the early 1700s to the mid- 
1950S. Historical records suggest that Martha’s Vineyard Sign Language 
came from an earlier sign language in England called Old Kent Sign 
Language. The language flourished on Martha’s Vineyard because 

of the unusually high number of deaf people who lived there and its 
acceptance among the hearing community. The fact that it was used 
both by hearing and deaf people alike for so long is a testament to the 
powerful social drive to connect through language. 


The semantic component of sign language involves the meaning of 
words. In the Saussurean tradition, words not only serve as symbols 
for things but as arbitrary symbols. For instance, the American Sign 
Language (ASL) word for sunlight is a small, one-handed, circular 
movement off to the side of the head. Then, the signer opens the 
hand toward his or her face. This seems fairly iconic, but there are still 
some arbitrary elements. For instance, why not have the hand directly 
overhead? Why not make a fist instead of a circle? Even with signs that 
are highly iconic, there are always arbitrary elements. 


When it comes to morphology and syntax, a myth about sign is that 
it basically borrows its grammar from spoken languages. There is no 
question that spoken languages influence sign languages, but sign has 
many unique grammatical features too. 


For example, Auslan is the imported sign language used by the 
Australian deaf community. It was brought to Australia by British and 
Irish deaf signers in the 19th century, and it differs from English in 
many ways. The past tense is one way: English adds the morpheme 
-ed to regular verbs to indicate an event happened in the past. 
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Auslan has no such marker, but instead it begins the sentence 
with a temporal statement and then describes an event. Take this 
example: “Week past, | wash my car.” 


Another myth about sign language is that it is all about the hands. This 
is not the case. For example, in 2012, New York City mayor Michael 
Bloomberg gave several press conferences about the administration’s 
response to Hurricane Sandy. At the podium, the mayor was joined by 
an ASL interpreter named Lydia Callis, who used animated and assorted 
facial expressions. These expressions were not just optional visual 
flourishes, like they are for spoken language. Instead, they were actual 
parts of the morphology, syntax, and semantics of ASL. In fact, the 
whole body is used in sign language. 


The final component is phonology, which, in speech, is determined 
by the physical configurations of the motor articulators, such as the 
lungs, throat, tongue, and lips. Phonology in sign is determined by the 
physical constraints of the hands, face, and body. The hands are the 
primary articulator. 


In the realm of sign phonology, a key figure is the pioneering linguist 
William Stokoe. In 1960, Stokoe published Sign Language Structure. 
The book was notable for three large reasons. First, it put an official 
name to the unique version of sign language used in America. It has 
been known as American Sign Language ever since. 


Second, Stokoe created a notation system for recording sign utterances 
in writing. Up to that point, no sign system had ever been reliably 
recorded on paper. Third, Stokoe’s book was the first to empirically 
demonstrate that ASL was an actual language, just as rich in structure 
as any spoken language. One of Stokoe’s main areas of focus was the 
phonology of sign. 


The phonology of spoken language concerns several parameters, 

such as shape of mouth, location of tongue, timing of vocal cord 
vibration, and so on. Stokoe showed that ASL was composed of similar 
parameters, but all in the manual modality. The big three are the 
shape, location, and motion of the hands. This means that for every 
sign, these three things systematically combine to produce distinct 
meanings. Since Stokoe’s publication, there have been a few other 
phonetic parameters added, like palm orientation and signals with 

the face. 


Development and the Brain 


The final pieces of evidence that have sealed sign as a real language 
came from the fields of developmental psychology and cognitive 
neuroscience. Developmentally, the native acquisition of speech 

and sign is remarkably similar. Verbal babbling and babbling with the 
hands both occur at around six months of age, and this is followed by 
non-sign pointing at about nine months of age. These typically lead 
to one-word speech and sign at about one year. By 18 to 24 months, 
signing and speaking children are stringing together two signs or 
words to form simple sentences. These simple sentences follow 
reliable word order rules. 
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Additionally, clinicians know from patients with brain damage that 
signers can have aphasia. Just like hearing aphasics, signers with Broca’s 
aphasia understand meaning, but they have difficulty producing basic 
phonological forms. Although the signing of Wernicke’s aphasics 
appears fluent, individuals with this form of aphasia have great trouble 
understanding and producing meaning. MRI scanners reveal that each 
type of aphasia has a neural profile of damage that looks a lot like its 
spoken counterpart. 


Findings like this have helped debunk one of sign’s biggest myths: 
Deaf signers and hearing speakers have totally different brains. 

In terms of neural mechanisms, language is language, largely 
regardless of modality. 


Tying back to one of this course’s themes, over evolution, the brain 
didn’t adapt to language; language adapted to the brain. That is what 
Morten Christiansen and Nick Chater theorize produced the tight fit 
between the brain and language in the first place. 


Running with this, we might conclude the following: Because hearing 
and deaf individuals are born with comparable domain-general neural 
architecture, both groups have created a language system that works 
according to those shared mechanisms. Although the superficial form 
clearly differs, signed and spoken languages are both products of the 
same language-ready brains. 


Sign Culture 


One final myth is the belief, held by many hearing people, that all 
deaf signers would rather use spoken language. The truth is that it is 
not so simple. For the first time ever, technological innovations like 
cochlear implants offer deaf children a chance to boost their hearing 
abilities, and this can make a huge difference in successfully learning 
to use spoken language. While most people in the deaf community 
are enthusiastic about such technologies, there is real concern 

that learning spoken language will come at the cost of implanted 
children not learning sign language. 
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Culture and language are inextricably tied. For deaf people, sharing 

a form of language that is perfectly suited to the people who use it 
produces an incredible sense of community. This can build a real love 
for the language. 


As much as any culture—and perhaps much more—the sign community 


is proud of their language, and they view it as inherently beautiful. 
Many hearing people are surprised to learn that, just like spoken 
language, sign language can be an art form. Deaf culture has its own 
revered poets, talented singers, and enchanting storytellers, all of 
whom exquisitely sign. Many expressions are unique to the language 
and culture, capturing things in a way that could never be expressed 
in speech. 


Suggested Reading 
Johnston and Schembri, Australian Sign Language (Auslan). 


Xu, Gannon, Emmorey, Smith, and Braun, “Symbolic Gestures and 
Spoken Language are Processed by a Common Neural System.” 


Questions to Consider 


What are some advantages and disadvantages of sign languages 
compared to spoken languages? Can you see the benefits 

of a hybrid form of communication that takes advantage of 
both modalities? 
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19. 
EMBODIED LANGUAGE: 
MIND IN BODY—BODY IN MIND 


The hands are closely tied to language. This lecture 
considers the rest of the body. The lecture focuses on 
answering this question: What can communication in 
its most embodied form tell us about language and 
the mind? 


The mind emerges from how the brain interacts with 
the environment through the body. In this way, the body 
isn’t merely a vessel for the mind. It’s actually part of the 
mind. This is the idea of embodiment, which has roots 

in a number of different fields, including philosophy, 
psychology, and neuroscience. 


Philosophy and Embodiment 


Starting with philosophy, the body has always been viewed as a central 
feature of the mind in many Eastern traditions, like Buddhism. 

For example, Zen Buddhism has long rejected the notion of the body 
as just a shell for the mind. In this form of Buddhism, body and mind 
mutually comprise the self. 


Until recently, the body has not received as much attention in Western 
philosophy. Most now credit the French phenomenologist Maurice 
Merleau-Ponty as the first Western philosopher to give serious 
attention to the body in human subjective experience. For Merleau- 
Ponty, the body serves as a constructive mesh between the brain and 
the environment, and this blurs the distinction between the inner mind 
and outer world. 


Psychology and Embodiment 


Merleau-Ponty’s ideas have had a large impact on scientific theories as 
well. This is most directly evident in the mid-2oth-century psychological 
work of the husband-wife team of James J. Gibson and Eleanor Gibson. 
Like Merleau-Ponty, the Gibsons saw the body as creating a mesh 
between the brain and the environment. 


They explored this interconnection through very naturalistic scientific 
studies in which people interacted with objects in everyday contexts. 

In these studies, they observed that people don’t seem to perceive their 
environment directly; instead, they experience it according to certain 
affordances. An affordance is an aspect of the environment that lends 
itself to a particular way of physically engaging with it. 


The Gibsons were not the only prominent psychologists to 
emphasize the body. Another was the developmental psychologist 
Jean Piaget. Piaget developed a theory called constructivism 

that emphasized how children actively constructed their 

knowledge over development. Piaget theorized that children act like 
small scientists who physically tinker with the world. 
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They use their bodies as tools to test hypotheses about things around 
them, like the physics of objects and the consequences of actions. 
Piaget’s work has been criticized on methodological grounds, but his 
central claim about how the body constructs knowledge has endured. 


Neuroscience and Embodiment 


In neuroscience, certain theories place a central importance of the 
body in neural processing. For example, many see simulation theory 
as a neural pillar of embodiment. In a similar way, Giacomo Rizzolatti’s 
work on mirror neurons is seen as support for embodiment in 
perspective taking. 


Before the 1990s, there were two competing views of the brain: One 
saw the brain as a general-purpose device like a computer, which 
computed and solved problems in a highly abstract and binary way. 
The other view of the brain saw it as a complex machine with many 
specialized parts for solving very particular problems. 


These so-called modules were functionally isolated from one another, 
which highlighted a sharp distinction between the body and higher- 
level thought. Both of these views waned in popularity during the rise 
of embodiment, but now they are making a comeback and adding 
important nuance to modern theories of embodiment. 


Critiques of Embodiment 


Despite the current popularity of embodiment, many scientists are 
pushing back on the idea, especially in its more extreme forms. One 
radical view of embodiment is that because humans have evolved 
minds in the ubiquitous context of bodies over millions and millions of 
years, all thought and all feeling is constrained by the body in some way. 
There are three big problems with this extreme view. 


The first critique is that embodiment is not an all-encompassing 
mechanism for the mind. Research in cognitive neuroscience has 
shown that not every concept, memory, or feeling is embodied. Some 
are represented in the brain in purely abstract ways. 
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The second critique highlights the functional limits of embodiment. For 
understanding concrete things—like an apple or a comb—it is obvious 
how neural embodiment is useful. Less obvious is how embodiment is 
useful for understanding abstract concepts like good and evil. 


The final critique of embodiment is best illustrated through an example 
from the documentary AlphaGo. It’s about the ancient Chinese board 
game Go, and it features a match between the 18-time world champion 
Korean player Lee Sedol and a computer program called AlphaGo. 


The program was created by Google’s DeepMind division in London, and 
it learned how to play the game through deep learning. AlphaGo was fed 
massive amounts of data from previous Go games while also training 
itself by playing actual Go experts. After training, it was 100 percent in 
control of its own moves during each game with Sedol. 


AlphaGo beat Sedol four games to one, and it displayed creative moves. 
If a computer can manage to accomplish this without a body, it seems to 
raise doubt about how much humans must use their bodies to think. 


Grounded Cognition 


Overall, the evidence suggests that some aspects of thought lend 
themselves best to an embodied form and other aspects are better 
suited for more abstract representations and processes. For example, 
the neuroscientists Bradford Mahon and Alfonso Caramazza present 
a hybrid view that they call grounding by interaction. 


The idea is that all thinking has an abstract component, but when 
possible, the way it becomes instantiated is through embodied 
perceptions and actions. In this way, abstract thought is grounded 
in how the body interacts with the environment. When viewed from 
this perspective, the mind has the best of both worlds: It can quickly 
and flexibly process abstract concepts without any limits while 

also materializing and exploring that abstract information through 
concrete bodily experience. 
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Embodied Language 


Given that the mind is both embodied and disembodied, it’s worth 
asking how this plays out in face-to-face communication. In other words, 
how do the arbitrary and abstract elements of language interact with 
the iconic and concrete expressions of the body? 


The key here is to recognize that language originally evolved within 
face-to-face contexts. In addition to the hands, the early forms of 
human language most certainly used other aspects of the body, 
including eye gaze, body posture, facial expression, lip movements, 
and tone of voice. In language’s natural face-to-face habitat, the 

body is an extremely important tool for communication. The main 
benefit of face-to-face interactions is that the body can help us 
understand what communicators mean. Brains are built for this sort of 
multimodal processing. 


For example, in the primate brain, the modalities of vision and audition 
synergistically interact to help differentiate important things like threats 
versus non-threats or mates versus non-mates. Language was built on 
top of these multimodal mechanisms, so it’s no surprise that humans 
depend on the body so much when we communicate. 


Emotional Expressions 


The psychologist Paul Ekman headed an ambitious research program 
investigating how facial expressions may universally display emotions 
across cultures. Although there is still debate, most scientists 

now generally accept that certain basic facial expressions—like 
happiness, anger, and fear—are universally used and differentiated 
across cultures, and they serve as reliable social signals for most people 
on the planet. 


However, emotional expression does not stop at the face. The cognitive 
neuroscientist Beatrice de Gelder is an expert on how emotion 
conveyed through the face and the body combine. Her work has found 
that the brain perceives emotional messages quickest when the face 
and body postures are emotionally congruent. When the face and 
body express the same thing but conflict with the emotional tone 

and content of speech, people go with what they see, not with what 
they hear. 


The Body and Pragmatics 


Pragmatics is all about assessing the social intentions of a 
communicator. The body is perfectly suited for that job. For example, 
take threat detection. Our fight-or-flight response is triggered by 
threatening stimuli, including aggressive postures and eye contact, 
angry facial expressions, and hostile tone of voice. 


According to the neuroscientist Joseph LeDoux, these threats are 
processed by two simultaneous pathways in the brain. He calls the 
first one the low road of threat processing. This low road is based 

on an evolutionarily ancient direct connection between sensory 
receptors, like the eyes and ears, and the amygdala, which is one of the 
oldest structures in the animal brain. The function of this pathway is 
to quickly alert the organism of immediate danger, and this happens 
within milliseconds. 
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At the same time, the high road of processing involves a much slower 
route, on the order of seconds. The high road involves the primary 
sensory cortices and the hippocampus. The high road creates a deeper 
and more thorough assessment of the threat in context. 


Suggested Reading 


Mahon and Caramazza, “A Critical Look at the Embodied Cognition 
Hypothesis and a New Proposal for Grounding Conceptual 
Content.” 


McGurk and MacDonald, “Hearing Lips and Seeing Voices.” 


Question to Consider 


Consider statements like “Follow your heart’s desire.” It seems 
natural to associate the body with emotion. Traditionally, feelings 
in the body have been contrasted to what is supposed to happen 
in the head, like perceiving, thinking, and reasoning. How does 
embodiment blur this distinction? 
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20. 
THE MULTILINGUAL MIND 


This lecture discusses the complexity of 
language in its multilingual form. It is helpful 

to understand this lecture’s terminology: The 
term multilingual refers to living with more than 
one language and regularly using them in daily 
life fluently. The term fluent refers to speaking 
or signing in a flowing manner. Finally, the 

term monolingual refers to being fluent in only 
one language. 


Multilingual Variability 


One widespread myth is that a bilingual brain is simply two 
monolingual brains rolled into one. This is incorrect. Although there 
are many similarities, bilingual and monolingual brains are different in 
some important ways. For instance, multilingual neural mechanisms 
are much more complex and variable than monolingual ones. 


This variability is caused by a number of factors, including how many 
languages a person speaks, the similarity among those languages, 
whether they are learned at home or in the classroom, and the amount 
of use each one receives. The biggest factor, however, may be age 

of acquisition. 


Age of exposure strongly predicts phonetic ability in a language. 

For example, Elissa Newport and colleagues have argued that many 
linguistic components are very fragile. Mastering them requires 
exposure to a language within the first 12 years of life. In addition 

to accent, there are sensitive components like complex syntax and 
inflectional morphology Guch as markers for past tense and plurals). 


However, Newport observes that other linguistic aspects—like 
vocabulary, simple syntax, and basic pragmatics—are relatively easy to 
learn even late in adulthood. Still, there is much variability in what late 
learners can handle. 


Multilingual Brains 


In 1997, Karl Kim, Joy Hirsch, and their team at the Sloan-Kettering 
Cancer Center in New York published a landmark study investigating 
how age of second language acquisition affects the language network 
in bilingual brains. The study used fMRI to compare differences in the 
Broca’s and Wernicke’s areas of people who learned second languages 
as infants or as adults. 
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Participants were put in a scanner and asked to internally generate 
silent sentences to activate neural networks of one language and 
then the other language. The main finding was that while Wernicke’s 
area showed no differences across languages for both early and late 
learners, Broca’s area did show a difference: For late learners, the two 
languages activated two separate subregions of Broca’s area, but for 
early learners, both languages activated an overlapping area. 


This pattern in Broca’s area makes sense given the dorsal stream 

of language processing. This part of the language network is where 
phonological forms of words are activated and executed. Patricia Kuhl’s 
theory of native language neural commitment is helpful here. Because 
the brain commits to phonemes of a language (or languages) early in 
life, it can use prime real estate in Broca’s area for the job. However, if 
a second language is learned after this neural commitment, it must rely 
on a different part of Broca’s area that is not as optimally suited for 
phonological production. 


Since Kim and Hirsch’s landmark experiment, subsequent neuroimaging 
studies have mostly confirmed these differences using a variety of 
languages and ages. Additionally, a recent meta-analysis shows that 

the later in life someone learns a second language, the more space is 
required in Broca’s area. 


Multilinguals versus Monolinguals 


One of the biggest differences between monolinguals and multilinguals 
is that multilinguals must build mechanisms for keeping track of their 
separate languages. The prefrontal cortex is constantly at work when 
we use language. It has to keep track of what was just said, and it must 
monitor what to say next. 


On top of that, it also has to suppress information that is distracting 

to the task at hand. This suppression is hard enough for monolinguals. 
Multilinguals must work extra hard because when they are speaking one 
language, their other language (or languages) must not get in the way. 


20. THE MULTILINGUAL MIND 


137 


Evidence for language coactivation and 
suppression in multilinguals comes from 
many different lines of research. This 
lecture now turns to four of them. 


Widespread 
Multilingualism 


Bilingual Aphasia The linguist 
François Grosjean 
estimates that 
more than half 


The first line of research involves 
bilingual aphasia. One distinctive, but 
not uncommon, problem in bilingual 
aphasics is pathological mixing of of the world’s 
languages within an utterance. It’s called population 
pathological mixing to set it apart from regularly uses 
voluntary mixing. more than one 


Voluntary mixing is perfectly natural, language. 
with one example being Franglais, a mix 
between French and English. Pathological 
mixing is where an aphasic can’t control 
swapping languages. 


In 2000, Franco Fabbro and his team of neurolinguists published 

a landmark case study of a patient who demonstrated pathological 
switching of his two languages: Italian and Friulan. (Friulan is a language 
spoken in northeastern Italy.) 


The patient frequently alternated entire sentences between Italian and 
Friulan, even when speaking to people who didn’t understand one of 
the languages. In this particular case, the patient was aware that he was 
making these switches, and it caused him great distress. 


After his symptoms got worse, and he started having difficulty walking 
and driving, he went to his doctor, who suggested an MRI to look for 
structural abnormalities in his brain. The results showed that he had 

a large tumor growing in the left inferior frontal gyrus. The damage 
was not quite in Broca’s area, which explained why he could still speak 
fluently in both languages. 
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The damage was to an area slightly anterior to Broca’s area. It was 
located in the prefrontal cortex’s executive control network, which is 
important for neural inhibition. In this patient’s case, damage to the 
left PFC made it impossible to stop alternating between languages, 
and it also produced more general deficits in social inhibition. 


Since this study, there have been other cases of structural damage to 
the left PFC and pathological code switching. As with Fabbro’s case 
study, these patients also demonstrated more general deficits in social 
inhibition. This has led many researchers to conclude that this is not 

a problem specific to language, but rather a more domain-general 
problem of executive control. 


Neurotypical Bilingual Brains 


A second line of evidence for the co-activation of multiple 

languages comes from research on neurotypical bilingual brains. 

In one simple but creative approach, the psycholinguist Albert Costa 
exploited the fact that some bilinguals have many shared vocabulary 
items across their two languages. These overlaps are called cognates. 


l F \— 


For example, take Spanish-Catalan bilinguals, who have an unusually 
high number of cognates. In Catalan, the word for cat is gat, and in 
Spanish, it’s gato. Costa had fluent Spanish-Catalan bilinguals look at 
pictures of things that either had cognates in the two languages or 
didn’t have them. The task was to name the objects in either Catalan 
or Spanish as quickly as possible. 


The prediction was that if the mental lexicons of Catalan and 

Spanish were both active when trying to name a picture, bilinguals 
should be slower to name objects that had no cognates. That’s exactly 
what Costa found. He interpreted this finding to mean that when 
bilinguals are searching for lexical items for a particular meaning, 

both languages are simultaneously active. 


The Other Side of Cognates 


The third line of evidence for multilanguage co-activation considers 
the other side of cognates. For all bilinguals, there are some words that 
randomly sound similar across their languages but have completely 
different meanings. 


For example, the Russian word for stamp is markka. A similar- 
sounding but different-meaning word in English is marker. What 
happens when a Russian-English bilingual hears the beginning of the 
words markka and marker? Do they activate both meanings in their 
heads momentarily? 


That was precisely the question asked by the psycholinguists 

Viorica Marian and Michael Spivey in a 2003 eye-tracking study. 

The experiment studied Russian-English bilinguals as they listened 

to a Russian speaker referring to different objects in Russian. Also 
involved was a computer screen that had an image of a stamp on 

the left side and a dry-erase marker on the right side. When subjects 
heard the Russian word for markka, their task was to press the side of 
the screen where the object was located—in this case, the left side. 
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This sounds like it should be a straightforward task, but the eye-gaze 
patterns suggested something more nuanced. When the stamp was 
paired with a marker, bilinguals looked at the marker about a third of the 
time. This means that there was competition between the phonological 
forms of the two words, markka and marker. In contrast, when the 
stamp was paired with a competing image that had a totally different- 
sounding English translation, like a 
candle, the bilinguals were three times 
less likely to look at that competitor. 


Language Attrition 
The experiment was done entirely guag 


in Russian, so there is no reason for It is not true that 
English to be active in the first place. 
Despite this, it wasn’t possible for the 
bilingual subjects to totally tune out 


once a person 
has mastered 


the irrelevant language, even when it a language, it is 

was disruptive. mastered for life. 
Without activity, all 

Neuroimaging Studies languages decay. 


This applies to 


The final line of evidence comes 3 person’s first 


from neuroimaging studies exploring 
how languages are activated and language, too. 
deactivated in neurotypical brains. Linguists call this 
The neural mechanism for this language attrition. 
process has been dubbed the bilingual 
language control network by the 
neurolinguists David Green and 

Jubin Abutalebi. 


In addition to the left prefrontal cortex, the network involves other 
areas outside the traditional language network, like the inferior parietal 
lobe, cerebellum, and thalamus. A particularly important area is the 
anterior cingulate cortex (ACC). The ACC is a subcortical brain structure 
involved in error detection and conflict monitoring. 
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Neuroimaging studies have shown that the ACC is heavily used in 
bilinguals when monitoring which language is most appropriate for 

a given communicative context. Interestingly, this region is most active 
for low-proficiency multilinguals, presumably because they are the 
ones who are most likely to make errors. 


The ACC and left PFC combine to perform two critical functions for 
multilinguals: They determine when to engage and when to disengage 
different languages. Both functions are important, but it turns out 
that disengaging requires much more brainpower. In a neuroimaging 
study in 2018, Karen Emmorey and colleagues investigated this 
control network. They found that turning off a second language 
requires much more effort than turning it on. 


Suggested Reading 


Costa and Sebastian-Gallés, “How Does the Bilingual Experience 
Sculpt the Brain?” 


Grosjean, Life with Two Languages. 


Question to Consider 


Decreased plasticity makes language learning harder for older 
people. Based on what you learned about neural inhibition in the 
prefrontal cortex, can you think of another mechanism for why late 
learners may struggle to master an additional language? 
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21. 
DOES LANGUAGE 
SHAPE THOUGHT? 


Answering the big question of 
how language makes us human 
requires the full power of the 
3-D framework. It is necessary 
to consider mechanisms and 
functions that operate on 
different timeframes and across 
multiple levels of analysis. 


Beginning with this lecture, this 
course uses the 3-D framework to 
explore two ways that language 
can shape who we are. One way 
considers how having any 
language molds various aspects of 
the mind, like emotion, personality, 
and thought. A second way 
considers how having a 

particular language—like English, 
Hebrew, or Navaho—influences 
minds specifically. 


The second, specific effect 

of language is much more 
controversial than the first. 
That specific effect is the focus 
of this lecture. 


The Whorfian Hypothesis 


Eskimo-Aleut languages have many words that describe snowy things. 
This diverse vocabulary was first formally discussed by the anthropologist 
Franz Boas in his 1911 work, the Handbook of American Indian Languages. 


For example, Inuit has a few different word stems for the term snow in 
addition to many morphemes that add nuance to the central meaning. 
Some variants describe snow blowing, slushy snow, hard snow, and so 
on. Even though many languages, like English, generate most of their 
grammatical complexity through syntax, other languages, like Inuit, 
put more complexity into their morphology. This observation was 
made controversial by a later claim from the budding linguist Benjamin 
Lee Whorf. 


In 1940, Whorf published a short paper called “Science and Linguistics” 

in which he took Boas’s observation a few steps further. He argued that 
possessing so many unique words for snow caused Eskimos to experience 
their world differently than people whose language has only a few terms 
for snow. Eventually, these ideas congealed into what is now referred to as 
the Whorfian hypothesis or the Sapir-Whorf hypothesis. The latter name 
reflects the contribution of Whorf’s teacher, the linguist Edward Sapir. 


Two Versions 


The hypothesis sprouted two versions, one strong and one weak. 

The strong version is called linguistic determinism, which is the idea 
that language enables or dictates thought. This view can lead to some 
extreme interpretations, including: If a person’s language doesn’t have 
an expression for something, the person can’t experience it. Taken to 

its logical extreme, this view leads to some absurd conclusions, with one 
being that babies don’t think at all until they learn language. 


Unfortunately, this extreme version distracts from the much more 
plausible version of the hypothesis, which is called linguistic relativity. 
This toned-down version claims that differences across languages 
simply influence differences in thinking. This is still hotly debated, but 
the debate is more about the nature and extent of this influence, and 
less about it being logically nonsensical. 
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A Snowy Argument 


The Eskimo snow example is the controversial topic that began the 
debate. One of the most well-known critiques was from the British- 
American linguist Geoffrey Pullum. Pullum made his argument 

first in an essay. In 1991, he turned it into a book: The Great Eskimo 
Vocabulary Hoax. 


One of Pullum’s main arguments was that, in terms of semantics, there 
is nothing special about how Eskimos refer to snow. They may have 
dedicated words to describe various snowy things, but other languages 
can do the same thing. For example, unlike Inuit, English does not have 
a single word for “snow drifting along the ground,” but English can still 
easily express that concept. 


Here is a reasonable counterargument: Might there be something 
special about having a single, dedicated word for a concept versus 
having many optional ways to express it? When snow is blowing 
sideways, there is no choice in Inuit to simply use a generic word for 
snow. Using the precise one is obligatory. 


In English, we have the freedom to say what we want about the snow. 
In the context of how habit and plasticity are related, this distinction 
between vocabulary that is optional versus nonoptional might make 
a big neural difference. 


Imagine a person taking the same route home from work every day 

for a year. Now compare that to the person randomly trying different 
routes on different days for that period. The habit of taking the same 
route nudges the brain to notice things about that particular route that 
one might miss with a random set of routes. This can be thought of as 
a type of neural commitment that guides the brain to experience and 
notice certain things. 


Honorifics 


The Japanese language features many components that deal with social 
relations. It has extensive vocabulary and morphology that highlight 
people’s places in the social world. In some cases, the exact same verb 
is marked by different suffixes according to one’s social status ina 
situation. The job of these suffixes is to make clear the social hierarchy 
between the person talking and the person listening. These are 

called honorifics. 


Honorifics are present in English too, but they are different in two 

big ways. First, they don’t affect verbs, and they’re not marked by 
morphemes. They’re usually standalone nouns or adjectives, like doctor, 
professor, or madame. The second reason is that the use of English 
honorifics is highly variable, especially in America. For example, some 
people insist on being referred to with a certain honorific much more 
than others. 


This relates to linguistic relativity because in addition to these linguistic 
differences, Japan and America have some big cultural differences. 
Respecting one’s place in the social order is an explicit value in Japan, 
whereas in America, it’s much more fluid. In fact, many Americans reject 
social hierarchies and actively try to disrupt them. 
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Value Systems and Language 


A relevant Whorfian question is this: Is it possible that these different 
value systems are influenced by the different honorific patterns of 
the two languages? At least on the surface, it makes sense how using 
honorifics may strengthen certain cultural habits of mind. 


However, a complication is this: How can we be sure that it’s the 
honorific structure of the language that is influencing the cultural 
value and not the other way around? Isn’t it equally possible that a 
culture that values a structured social order would create a linguistic 
system that reflects that order? This is the gist of the argument that 
John McWhorter presents in his 2014 book The Language Hoax. 
McWhorter’s critique is that language merely reflects culture, and it 
is culture that does all the heavy lifting in influencing thought across 
different groups. 


To explore this critique, a well-known example from the realm of 
language and spatial cognition is helpful. The cognitive scientist Lera 
Boroditsky is fond of doing the following demo in lecture halls around 


America: She asks the audience to close their eyes and point due north. 


Audiences typically have trouble with this seemingly simple task. 


Contrast this ignorance with the behavior of an Aboriginal community 
in northern Australia. The community lives in Pormpuraaw, on the 
western edge of Cape York, and their primary language is called 

Kuuk Thaayorre. The language is different from English in how it 
describes spatial relations. Rather than relying on relative spatial terms 
like left and right, it uses absolute spatial terms like north, south, east, 
and west. 


Speakers of Kuuk Thaayorre also happen to be very good with cardinal 
directions. If asked to close their eyes and point due north, they have 
little problem. The linguistic anthropologist Stephen Levinson has 

a video of a girl in Pormpuraaw who correctly identifies directions 
even after being blindfolded and spun around five times. The girl is 
among many in the community who can do this. 
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To systematically test these spatial abilities, Levinson and colleagues 
have conducted several field studies in Pormpuraaw (and in other places 
where people use absolute directions in speech). As a comparison 
group, they also tested Dutch speakers in the Netherlands, who use 
relative spatial terms, as is the case in English. Their findings imply that 
because Kuuk Thaayorre speakers habitually use absolute terms for 
space, they encode that information in a compass-like way. 


McWhorter might argue that the landscape of Pormpuraaw is ideally 
suited for cardinal directionality, and their language simply reflects 
this fact. In contrast, Westerners like the Dutch live in a landscape 
where absolute directions may not be the best way to describe 

and categorize space. After all, with so much of Western life operating 
indoors, large-scale landmarks are less prominent and can’t always 

be counted on. This may naturally lock in relative spatial terms, like 
left and right, as the most useful form of language in these cultures. 
Perhaps it is the physical environment that affects thought and not 
the language. This possibility signals trouble for linguistic relativity. 


Color 


This lecture concludes with a look at color, and specifically this 
Whorfian question: Do people see blue because they have a word for 
blue? Not all languages have words for every color. Even some standard 
color words in English are relatively new to the language. For example, 
the term blue made its appearance relatively recently, only after the 
arrival of certain other color words. 


Debi Roberson and her team at the University of Essex studied color 
words of a community of cattle herders in Namibia. They found that 
the herders’ language, Herero, does not have as many color terms as 
does English. Whereas English has 11 basic color words, Herero has 
only 5. For example, the single Herero word zuzu subsumes a range of 
darker colors that English speakers would identify separately. 
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Additionally, the Herero speakers do not sort colors the same 

way as English speakers do. It appears that the two groups have 
categorical differences in how they perceive and organize colors. 
However, keep in mind that this does not mean that the differences in 
color vocabulary cause the differences in categorization. 


Suggested Reading 


Haun, Rapold, Janzen, and Levinson, “Plasticity of Human Spatial 
Cognition.” 


McWhorter, The Language Hoax. 


Questions to Consider 


What sort of experiment do you think would help test the causal 
claims made by linguistic relativity? Can you think of a way to 
control for variables like physical environment and cultural 
differences—and then test the direct effects of different languages 
on thought? 
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22. 
DOES CULTURE 
SHAPE LANGUAGE? 


According to the linguist John 
McWhorter’s book The Language 
Hoax, culture precedes language and 
uses it to produce the diverse ways 
of experiencing and living in the 
world. In other words, culture is the 
agent for creating differences, and 
language is just a tool. This is a bold 
position that raises an important 
question: Is it really the case that 
language exerts itself only as a 
vehicle for culture? 


The Color Blue 


English speakers can capture about 15 percent of the visible color 
spectrum within the word blue. This range includes very light hues, like 
powder blue, to very dark ones, like royal blue. 


However, in Russian, matters are different. There are two categories 
of blue that roughly mean “light” and “dark.” They are guloboy and 
siniy. The distinction between guloboy and siniy is categorical 

and obligatory. They are actually separate colors, like purple and red 
in English. There doesn’t seem to be an obvious cultural reason for 
this linguistic difference. 


This presents an opportunity to see if linguistic relativity can operate 
outside the influence of culture. In one well-known experiment, 

a team led by Jonathen Winawer measured Russian speakers’ speed at 
differentiating colors within and across these categorical boundaries. 


They presented Russian speakers with a triad of color squares on 
a computer screen. The top color matched one of the bottom 
two colors—the target—but not the other color—the competitor. 
The task was to ignore the competitor and find the target color as 
quickly as possible. 


The main finding was that Russian speakers were 124 milliseconds faster 
at matching the top color to the bottom target when the competitor 
was from a different color category. This advantage was not present 
for English speakers who did the same task. For English speakers, there 
is only one general category of blue, not two. Additionally, for Russian 
speakers, obligatorily distinguishing colors in speech actually helps to 
visually distinguish them in perception. 


It seems unlikely that these findings are the product of cultural 
mechanisms. English and Russian speakers both use the same full 
spectrum of colors in everyday life, and the distinction between light 
and dark blue does not have any special significance in Russian culture. 
Additionally, language areas in the brain are active during color 
processing, which suggests a likely linguistic mechanism. 
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Two Mechanisms 


Other studies help to tell a more complete story about the 

relationship between language and the mind. In 2011, Veronica Kwok 
and Li Hai Tan led a study that investigated how exposing Mandarin 
speakers to new words for colors might induce plasticity in visual areas 
of the brain. In the experiment, participants were presented two shades 
of green, and then they were taught two made-up labels for these 

two shades: ang for light green and song for a slightly darker green. 

The same thing was done for two shades of blue, with different made- 
up words. 


After just three days, there was an enlarged cluster of gray matter in 
a region of the visual cortex specialized for color. The findings show 
that the brain’s inherent plasticity allows for structural modifications 
even after short bouts of practice. 


Another experiment, published in 2017, was led by Anna Franklin and 
Alice Skelton. The subjects of this study were four months old. The 
study sought to answer this question: Before being fully immersed in 
language and culture, do preverbal infants have an innate bias in how 
they process color? 


There were two interesting findings. The first was that infants made 
color distinctions that aligned remarkably well with categories that 
commonly occur in the world’s different color lexicons. This is notable 
because color categorization in this experiment occurred long before 
children acquire words for color. 


The second interesting finding was that infants’ color distinctions nicely 
corresponded to two neural subsystems in the brain’s processing of 
color. The primary visual cortex (V1) is organized in a highly structured 
fashion, with different dimensions of color requiring specialized 
computational mechanisms. These mechanisms appear to act like 
biological fault lines, and infants seem to use them to make categorical 
distinctions between hues. 
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The research on infants by Skelton and her colleagues gives us insight 
into the evolutionary mechanisms of color vision. These foundations 
are ancient—much older than language. This suggests that the brain’s 
architectural and computational constraints for vision were the impetus 
to the world’s color vocabularies in the first place. In other words, 
because our eyes naturally differentiate certain colors, our language 
systems evolved to name them. 


In combination with these basic evolutionary fault lines for color 
differentiation, experience with language over development often 
accentuates these breaks. Sometimes it can alter them. This was 
the case with the variation of color terms across Russian and 
English speakers. 


Evolution gives humans a genetic head-start in color processing. 

This combines with the brain’s inherent plasticity, which allows social 
mechanisms over development to either deepen those pathways or 
forge new ones based on unique experience. This hybrid way of thinking 
about linguistic relativity is gaining more and more support from 
cognitive scientists. 


A Control Knob 


In 2017, Terry Regier proposed a helpful way of reconciling how 
universal foundations for cognition—like innate color biases—can 
coexist with the Sapir-Whorf hypothesis. Regier’s solution was to 
cast linguistic relativity as a type of Bayesian probabilistic inference. 
The idea is that when we are sure of something, we don’t need the 
help of language, but as uncertainty grows, so does our reliance on 
linguistic categorization. 


Regier has a catchy name for this mental process: He calls it the 
cognitive control knob. When basic perceptual information is sufficient, 
people don’t need to turn the knob. As perceptual certainty goes 
down, people turn up language to give an extra boost. 
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Regier’s idea is a classic domain-general approach to understanding 
the mind. At its core, the mechanism is basically a Bayesian probability 
analysis: When a person is uncertain about something, how does he 
or she weigh different information or cues to constrain decisions? 
That is an extremely general process. It applies not just to language 
cues but to all possible constraints on decision making. 


Cultural Relativity 


We can think of culture as a constraint, too. This is how the researcher 
and professor Daniel Casasanto views it. Casasanto likes to relate 
linguistic relativity to what he calls cultural relativity. 


By cultural relativity, he means something similar to Whorf’s 
linguistic relativity. He explains the idea in this manner: “Certain 
aspects of people’s thinking vary relative to the cultural 
practices they engage in.” In one of his favorite examples, he 
talks about metaphors for space being used to conceptualize 
the passage of time and how different cultural practices lead to 
different metaphors. 


First, he sets this up by pointing out that people across 

all cultures use spatial metaphors to talk about time. For 
example, in English, the past is behind us, and the future is in 
front of us. 


Casasanto makes an interesting observation: The sagittal axis—which 
refers to the front and behind—is nearly universal for spatial metaphors 
for time. However, no known language ever verbally refers to time on 
the lateral axis, which refers to left and right. In English, one would 
expect to hear “Monday comes before Tuesday,” but not to hear 
“Monday is to the left of Tuesday.” 


A seminal experiment demonstrated this. In 1991, the cognitive scientist 
Barbara Tversky had more than 1,000 English-speaking and Arabic- 
speaking children and adults place stickers labeled with the terms 
breakfast and dinner relative to a sticker labeled with the term lunch 
on the middle of a table. 
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The English-speaking subjects were much more likely to place the 
breakfast sticker to the left of the lunch sticker and the dinner sticker 
to the right of it. The Arabic speakers did the opposite. Arabic is read 
and written from right to left, not left to right like English. It appears 
that the cultural convention of reading and writing has caused the 
two groups to conceive of the arrow of time differently. 


A problem remains: Correlation does not equal causation. To pin 

down causality, Casasanto and his colleague Roberto Bottini devised 
an experiment with Dutch speakers. Like English, the Dutch read and 
write from left to right. In the experiment, subjects were asked to read 
phrases that had a common spatial metaphor for time, like “a year 
earlier.” The task was to press a red button if the phrase was about the 
future and a blue button if it was about the past. 


The red button was situated to the right side of the screen, and the blue 
button was on the left. (There were also trials that counterbalanced 

the colors to rule out color as a cause.) Importantly, the experimenters 
never mentioned the location of the buttons, only the colors. This was 
to ensure that subjects didn’t figure out that the location of the buttons 
was even relevant to the experiment. 


The main finding was that subjects were faster to press the buttons 
when they spatially matched the flow of time. For example, when they 
read “a year earlier,” they would press the button when it was on the 
left faster than when it was on the right. This was flipped with phrases 
about the future, like “a year later.” 


Some subjects were asked to read sentences that were mirror reversed, 
which meant they had to go from right to left, not left to right. 

This is initially hard to do, but it becomes easy with practice. Even after 
just five minutes of this practice, the spatial effects were completely 
reversed. Now people were fastest when pressing the left button for 
the future and the right button for the past. 
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This is a nice example of the usefulness of the experimental approach. 
Because the same subjects showed two different patterns, we can rule 
out that language or other cultural factors were at play. Those things 
were held constant, and the only thing that differed across conditions 
was reading direction. 


Cultural Values 


Another relevant study was performed in 2001 by Takahiko Masuda 
and Richard Nisbett. They explored a well-known cultural difference 
between Japanese and American people: Japanese culture focuses 
more on how individuals collectively fit into a social context, whereas 
American culture focuses more on individual autonomy. 


The experiment asked participants from both countries to watch 

a short video of an aquarium scene. In the foreground, there was 

a large central fish flanked by some small fish swimming past some 
seaweed. There were also some other peripheral details, like shells, 
crabs, and bubbles in the background. 


After watching the video twice, both groups were asked to verbally 
recall everything they saw in the video from memory. The main finding 
was that Japanese speakers excelled at holistically remembering 

the scene, recalling details about stationary objects and smaller 
creatures in the background much better than Americans. However, 
there were no differences in how both groups remembered the large 
focal fish or other moving objects that were more prominent. 


These findings were taken as evidence that basic cognitive processes, 
like attention and memory, are influenced by cultural values 
regarding the holistic analysis of a situation. Japanese culture values 
how an individual’s behavior fits into a larger context, so that’s 

what the Japanese subjects noticed and remembered from viewing 
the scene. 


Suggested Reading 


Kwok, Niu, Kay, Zhou, Mo, Jin, So, and Tan, “Learning New Color 
Names Produces Rapid Increase in Gray Matter in the Intact Adult 
Human Cortex.” 


Regier and Xu, “The Sapir-Whorf Hypothesis and Inference under 
Uncertainty.” 


Questions to Consider 


Running with Regier’s cognitive control knob analogy, can you think 
of examples in your life when language or culture—or both—played 
more or less of a role in how you experienced something? Did these 
instances involve different degrees of uncertainty? 
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23. 
THE BENEFITS OF 
BILINGUALISM 


This lecture focuses on answering a specific question: 
What occurs as a result of multilingualism? To answer 
this, the lecture first makes a distinction between the 
intended function of speaking more than one language 
and its many unintended consequences. 


The functional purpose of multilingualism is clear: to 
increase the number of people one can effectively 
converse with. However, as the eminent biologist 
Nikolaas Tinbergen points out, the fact that a behavior 
is designed for one thing doesn’t preclude it from doing 
other things. 


The Prefrontal Cortex 


Speaking multiple languages requires a powerful prefrontal cortex 
(or PFC), presumably to control and monitor which language is being 
used at what time. This role of the PFC is a classic domain-general 
skill; it’s not specific just to language. Given that, might this constant 
linguistic flexing of the PFC affect other aspects of cognition too? 


This was the question that made Ellen Bialystok of York University 

in Canada famous. In her 2001 book Bilingualism in Development, 
Bialystok introduced the general public to the idea that being bilingual 
enhanced cognitive control in children and adults. Cognitive control 

is a metacognitive ability of the PFC that involves deploying attention 
to what is relevant while ignoring what is not. 


Bialystok cleverly used a well-known measure of cognitive control 
called a Stroop task. Stroop tasks require one to focus on one 
dimension of a stimulus while actively ignoring another dimension. 


For example, imagine seeing the word blue written on a computer 
screen. Also imagine that sometimes it is written in a blue font, but 
other times it is written in a red font. Finally, imagine that the task is to 
identify the color of the font—red or blue—while ignoring the actual 
meaning of the word. 


On average, people are much slower at doing this when the word and 
font differ. Bialystok found that compared to monolinguals, bilingual 
children and adults showed much smaller timing differences, suggesting 
that they could more easily focus on the relevant piece of information 
without getting distracted by the irrelevant piece. 


One explanation for these findings is that because bilinguals can 
have up to a lifetime of experience activating and suppressing 
languages—and switching back and forth between them—they are 
constantly flexing their cognitive control muscle. This allows them 
to become very good at using cognitive control for all sorts of 
metacognitive activities—not just ones related to language. 
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Bialystok and her colleagues have also found that multilingualism 
may offer protective effects for normal age-related memory loss. 
For example, in one study, Bialystok reported that memory loss is 
delayed by four years in multilinguals compared to monolinguals. 


Personality and Multilingualism 


The pioneering sociolinguist Susan Ervin-Tripp was one of the first 
researchers to empirically study how people change when they speak 
different languages. Ervin-Tripp had French-English bilinguals perform 

a well-known psychological assessment called the thematic apperception 
test, or TAT. The TAT involves pictures of ambiguous scenes, like a man 
pulling away from a woman’s embrace while the woman looks directly 

at the man. These evocative pictures are designed to produce diverse 
stories about what is happening in the scene, what people in the scene 
are thinking and feeling, and what is going to happen next. 


One group of bilinguals was shown a set of pictures and asked to tell 

a story about them in English. Then they were shown the same set again 
(in a different order) and asked to tell stories about them in French. In 
asecond group, the bilinguals did the same thing, but they started in 
French and then switched to English. 


The main findings were that when the bilinguals were speaking French, 
they had more descriptions that mentioned verbal arguments, but when 
they were speaking English, they had more themes that included physical 
violence. Interestingly, when talking in English, people were also more 
likely to describe the women in the scenes as being supportive as wives, 
helping their husbands achieve their goals. 


More recently, Michéle Koven of the University of Illinois has done some 
interesting ethnographic analyses of French-Portuguese bilinguals 
speaking their two languages. According to Koven, when a bilingual 
speaks in one language, that language reactivates all the past experiences 
that are unique to speaking that language. The past experience becomes 
temporarily part of the present identity. 


Decision Making 


Another interesting area of study is how bilinguals make decisions when 
processing emotionally charged information in each of their languages. 
We know from psychologists like Nobel Prize-winner Daniel Kahneman 
that emotion can bias people to make irrational decisions. 


How does language fit into these decisions? This was the question 
asked by Boaz Keysar and his students at the University of Chicago. 
In 2012, they published a paper: “The Foreign-Language Effect: 
Thinking in a Foreign Tongue Reduces Decision Biases.” 


The researchers performed several experiments that involved different 
forms of risk-taking and different bilingual languages. They led Keysar 
and his team to conclude that a foreign language provides greater 
cognitive and emotional distance than a native tongue. This distance 
allows people to do a more objective analysis of information during 
decision making. 


This distancing mechanism has fascinating philosophical implications. 
Consider this classic ethical question: If you could throw a switch to 
divert a trolley car from one track where it will kill five people to a side 
track where it will kill only one person, would you pull that lever? 
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Albert Costa and his colleagues asked a variation of this question in 

a 2014 experiment involving various types of bilinguals, including English/ 
Spanish in the US, Korean/English in Korea, English/French in France, and 
English/Hebrew in Israel. Replicating previous research, subjects in 

all languages were much less likely to choose the utilitarian option of 
pulling the lever, which from a purely rational and objective point of 
view saves the most lives. 


Interestingly, people were particularly reluctant when doing the task 

in their native languages: Only 20 percent said they would pull the lever. 
They were much bolder when doing the task in a second language. 

On average, they were more than 50 percent more likely to flip 

the switch. 


These findings further suggest that using a foreign language helps 

to cognitively distance people from emotionally charged issues. 
Additionally, psychotherapists have long noted that their bilingual 
patients often work through hard emotional problems better in their 
non-native language than their native one. 


Social Skills 


If learning multiple languages gives people access to two different 
worlds, could exposure to more than one language enhance certain 
social abilities in multilinguals? For example, consider perspective taking. 
Monolingual children often grow up with uniform linguistic common 
ground: If everyone they know speaks the same language, it can feed 
the belief that they all share something major in common. 


Growing up with multiple languages is different. Having linguistic variety 
may cause children to work harder to keep track of what is mutually 
known and relevant at any given time. The cognitive science team of Dan 
Sperber and Deidre Wilson theorize that this mechanism of relevance is 
the key to success in all social communication. When you are exposed to 
one language, one major type of common ground is relevant, but when 
you are exposed to multiple ones, relevance is much more fluid. 
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Zoe Liberman, Katherine Kinzler, and their colleagues have explored 
these issues with children and infants. For example, in one 2015 
experiment, four- to six-year-olds growing up with just one language 
were compared to children exposed to multiple languages in a game 
that required social perspective taking. 


The game involved an adult speaking about some objects that the adult 
and child could mutually see and some other objects that only the child 
could see. If children were considering the perspective of the adult 
speaker, they should tune out information that only they could see. 
That is where common ground was absent. They should perk up when 
communication shifted to objects that they could both see—that is, 
when common ground was present. 


The main finding was that monolinguals looked at the two objects 
equally, whereas children exposed to a second language looked primarily 
at the object that both could see, suggesting a greater sensitivity to the 
adult’s perspective. The research team has since gone on to replicate 
this basic finding in 14- to 17-month-olds, so this boost to perspective 
taking starts early. 


Suggested Reading 
Bialystok, “Bilingualism.” 


Keysar, Hayakawa, and An, “The Foreign-Language Effect.” 


Question to Consider 


Even within a single language, different social contexts require 
different ways of communicating. Consider how you talk differently 
around your family, friends, coworkers, and strangers. How might 
going back and forth among these pragmatic contexts shift aspects 
of the mind even within a single language? 
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HOW LANGUAGE 

MAKES US HUMAN 

The mind is an emergent property produced by 
a synergy of the brain, body, and environment. 
Language is a powerful tool in this emergence. 


24. 


5 


This lecture explores the extent of that powe 


? 


particularly how language is linked to mathematics 


logic, storytelling, 


, and metaphor. 


categorization 


i l 


oye 


The Mathematical Mind 


Some people think about mathematics and language as totally 
separate things. However, mathematics is deeply grounded in language. 
In the mid-1990s, cognitive neuroscientist Stanislas Dehaene published 
a book called The Number Sense. He explained that humans have two 
main mechanisms for mathematic ability. 


The first mechanism, called the approximate system, is an innate 
domain-general skill that comes as standard equipment for human 
infants and is even shared across many other species. The system can 
make basic numeric distinctions in an approximate fashion without 
doing formal counting. 


This is different from the second mechanism, called the exact system, 
which is unique to humans starting at about age three. The exact 
system uses symbolic processing to understand numbers. For example, 
only ahuman can differentiate 59 dots from 60. Only a human is 
capable of counting that high. 


This second mechanism taps into the symbolic property of language. 
Math and language are thoroughly connected in the brain. Using 
functional neuroimaging, Dehaene and others have shown that when a 
person solves basic math equations, many parts of the brain’s language 
network are activated. Disrupting that network with neurostimulation 
disrupts the ability to do that math. 


This disruption is most extreme in cases of aphasia. Often, aphasics 
lose the ability to do even basic arithmetic, but their ability to do 
approximate estimations remains. 


Logic 


Asecond major product of language—with a close relation to 
mathematical thought—is the capacity for logic. Logic is a cornerstone 
of Enlightenment thinking, and just like math, it is firmly grounded in 
the symbolic properties of language. 
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However, logic may not always be a good thing. The poster child 
for logic in 1921 was a young Ludwig Wittgenstein. Early in his life, 
Wittgenstein embraced a view that came to be known as logical 
positivism, which held that if logical analysis cannot be used to 
solve a philosophical problem, it is not worth solving. 


Later in life, Wittgenstein started to see the dangers of logic. Here 
is a quotation from 1953 in Wittgenstein’s posthumously published 
Philosophical Investigations: “Philosophy is a battle against the 
bewitchment of our intelligence by means of language.” 


This is classic Wittgenstein, full of nuance and double meaning. 

Here is one read on the quotation: When it comes to human thought, 
language is both a disease and the cure. On one hand, philosophers 
can use everyday language to identify and investigate important 
philosophical problems. 


On the other hand, we can be misled by logical propositions of 
language, such as “If x, then y.” Wittgenstein came to believe that 
these propositions can create pseudo-problems and provide answers 
to questions that have no basis in reality. That is what philosophy 
must fight against. 


Storytelling 


A third major use of language is to construct narratives. Telling stories 
is a signature feature of being human. Humans tell tales both grand 
and small. Consider the powerful narratives driving human civilization. 
Big-picture historians, like Yuval Noah Harari, argue that it was the 
narrative and imaginative power of language that built human societies 
over the past 70,000 years. 


For example, in his ambitious book Sapiens: A Brief History of 
Humankind, Harari argues that humans have used language to create 
highly useful fictions that give artificial meaning and structure to our 
world. Take one of our most useful and long-lasting fictions: money, 
which is just an idea based on linguistic agreement among people. 
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People also create narratives on much smaller scales. For example, 
people create the stories of their lives. Narrative identities are 
extremely important, and fostering them can help people have 

a meaningful existence. Sometimes this means telling a story of 
one’s life that is not entirely based on reality. 


For some people, telling positive stories is much harder than telling 
negative ones, and this can cause much psychological pain. On a larger 
scale, fictions at the global level have done incredible damage over 

the course of history. Humans are natural storytellers, and we must 
accept that, big or small, these tales can take us to some good places. 
They can take us to some scary ones as well. 


Categorization 


A fourth potent function of language is its ability to carve up the world 
into categories. In a notable psychology study related to this, an MIT 
research team led by Leonard Lee went to real bars in Boston and had 
subjects do taste tests of two beers: Samuel Adams and what they 
called MIT Brew, which was just Samuel Adams plus several drops of 
balsamic vinegar. 


When knowing nothing about the secret ingredient, subjects chose 
the MIT Brew 60 percent of the time. However, when they were 

told the secret ingredient right before they took a sip, that number 
plummeted to about 20 percent. Even though people drank the 
exact same thing in both conditions, naming vinegar as an ingredient 
left a bad taste in people’s mouths. 


The ability to categorize is certainly not unique to humans, but 
language greatly enhances categorization because it explicitly marks 

a category. That is what words do: They signal what they are and what 
they are not. The word dog conjures up thoughts of features like fur, 
tails, and barking, and it inhibits thoughts of features like feathers, 
wings, and chirping. 
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Another powerful function of words is to create new categories 
where none existed. For example, the Dictionary of Obscure 

Sorrows is a website and YouTube channel created by the artist John 
Koenig. His dictionary is made up of new words that capture unusual 
but oddly familiar categories of experiences that—at least in English— 
have no name. One example from his work is the word sonder, which 
means “the realization that each random passerby is living a life as 
vivid and complex as your own.” 


Even though categorical thinking is extremely useful, it is a double- 
edged sword. Linguistic categories can exacerbate differences. 
They can promote harmful stereotypes, and they can deceive us 
into thinking the world is simpler than it really is. As with the other 
products of language, we must be clear-eyed in taking the good with 
the bad. 


Metaphor 


This lecture’s final major function of language is metaphor. 

Upon close examination, it may be surprising how often we use 
metaphors in daily life. In 1980, George Lakoff and Mark Johnson 
published a splashy book titled Metaphors We Live By. 


The book described the ubiquity of conceptual metaphors in everyday 
language. By showing their prevalence, Lakoff and Johnson made 

the case that metaphors are much more than just artistic flourishes. 
They are actually a fundamental part of human thought. 


Metaphor makes the abstract concrete, and this breathes life 

into language. For example, consider Wittgenstein’s statement: 
“Philosophy is a battle against the bewitchment of our intelligence by 
means of language.” This captures the imagination much better than 
saying, “Philosophers must recognize that language can confuse us, 
and they need to work hard to ensure that never happens.” 
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Another function of metaphor is to extract specific elements of 
something. If categories are good at drawing boundaries between 
different things, metaphors excel at highlighting similarities. 

By using the familiar word battle, Wittgenstein accentuates that 
philosophy is a process that can be violent, protracted, heroic, winnable, 
and painful, just to name a few elements. These connections help us to 
structure thought in targeted ways. 


Additionally, metaphor is a fount of creativity. It bubbles up in every 
possible place: music, law, art, poetry, film, literature, science, politics, 
and so on. Mapping one thing onto something similar yet different is 
a vehicle for seeing the world in a different way. 


However, therein lies its biggest downside. Metaphor is never a perfect 
vehicle. Something is always added, and something is always lost. 

This is a big problem for anyone using metaphor to try to capture the 
truth about the world. Perhaps metaphor is simply the quintessential 
manifestation of minds that have language—minds that understand one 
thing by mapping it onto another. 


Suggested Reading 
Clark, Being There. 


Dehaene, Spelke, Pinel, Stanescu, and Tsivkin, “Sources of 
Mathematical Thinking.” 


Lakoff and Mark Johnson, Metaphors We Live By. 


Question to Consider 


Consider this metaphor: Language is a double-edged sword. 
Building on the five examples from the final lecture, can you think 
of any other examples of how language helps and hurts humans? 
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